Secrets Behind the Ronnie James Dio Hologram: From Motion Capture to Pepper's Ghost | Revolver

Secrets Behind the Ronnie James Dio Hologram: From Motion Capture to Pepper's Ghost

Eyellusion's Chad Finnerty dives deep into the art and technology of bringing a late, great metal icon back to the stage
ronnie james dio hologram, Stephanie Cabral
photograph by Stephanie Cabral

"Ronnie's legs just arrived this morning."

That's what Wendy Dio, widow and manager of revered vocalist Ronnie James Dio, says when she greets Revolver in the green room at the Nate Holden Performing Arts Center in Los Angeles. Apparently, the bottom half of Dio's hologram had been stuck in Europe, where it completed a short tour in 2017. Luckily, it's just rolled in on what Wendy describes as "the slow boat from China."

Upstairs in the concert hall, we're treated to an exclusive preview of the latest Ronnie James Dio hologram. As images of dragons, fire and sword-wielding knights explode across a digital screen, a live band featuring Dio mainstays Craig Goldy (guitar), Simon Wright (drums) and Scott Warren (keyboards) — plus new bassist Bjorn Englen — rip through a handful of the late singer's greatest hits. In the center of at all, the ghostly visage of Heaven & Hell–era Ronnie appears onstage, gesturing vividly to "King of Rock N' Roll" and an epic medley of "The Last in Line" and "Holy Diver." The vocals are a recording, of course — from live shows, not the studio albums — but the effect is uncanny.

In between Dio songs, former Judas Priest vocalist Tim "Ripper" Owens and former Lynch Mob vocalist Oni Logan come out to perform a stunning rendition of Rainbow's "Stargazer." As it turns out, the upcoming Dio Returns tour won't just be an RJD hologram "singing" the classics — it will also feature real live vocalists Owens and Logan belting out more classics while the hologram gets a breather. If the demonstration we see is any indication, it'll be an impressive show. But it says a lot about Ronnie James Dio that it takes two live vocalists and a hologram to replace him.

The Dio hologram was created by a Los Angeles–based entertainment company called Eyellusion. They debuted an early version at Germany's Wacken Festival in 2016; a second version did the aforementioned Euro tour the following year. The version that's about to tour the U.S. is the Dio hologram's third incarnation. For Chad Finnerty, Eyellusion's Director of Creative Development, it's the culmination of years of hard work. "To say the least," he says with a laugh. "I'm literally shipping the last of the files today, which is very nice."

After seeing the Dio hologram in action, we asked Finnerty to give us the rundown.

dio hologram 2019, Stephanie Cabral
photograph by Stephanie Cabral

IN LAYMAN'S TERMS, WHAT IS A HOLOGRAM?
CHAD FINNERTY
What we actually present is not technically a hologram. It's called a hologram, and it's probably the closest thing to a hologram these days, but it's actually done with a technique called Pepper's Ghost. It's a reflective screen that bounces an image onto a see-through screen, giving the illusion of a hologram.

SO IT'S TWO-DIMENSIONAL RATHER THAN THREE-DIMENSIONAL?
Yeah, but here's the thing: What we create is made from 3D models of the person. So it could be three-dimensional if the technology would allow it to. If you were wearing virtual-reality goggles, it would absolutely be three-dimensional. The problem is, the technology hasn't caught up to where you could be hanging out with your buddies and seeing this hologram together, wearing VR goggles, without bumping into each other.

PEPPER'S GHOST TECHNOLOGY HAS BEEN AROUND SINCE THE CIVIL WAR, HASN'T IT?
Absolutely. It's an illusion — a magic trick. In our case, it's an Eyellusion. [Laughs] Our company is not really centered around technology, because as you said, this technology has been around for a while. We're focused on the content. We're trying to create it so it can be ported to pretty much any platform that exists. The real magic behind Eyellusion is creation of the shows — creating a spectacle for the audience.

AS FAR AS I KNOW, THE TUPAC SHAKUR HOLOGRAM AT COACHELLA IN 2012 WAS THE FIRST EXAMPLE OF A DECEASED MUSICIAN ONSTAGE IN A LIVE CONCERT SETTING. WAS THERE A SPECIFIC BREAKTHROUGH IN THE TECHNOLOGY THAT ALLOWED THE TUPAC HOLOGRAM TO BE POSSIBLE?
I think the breakthrough was on the visual effects side of things. For many, many years, creating a digital human was a very difficult thing to do. It's still a very difficult thing to do, but the technology that we use as artists has gotten much better over the years. For example, we can create much more realistic eyes, skin, the movement of the hair — this is all made up of ones and zeroes. It's all in the computer. That's the breakthrough. But the use of Pepper's Ghost in a concert setting was also something that hadn't been done, so I guess that would also be a breakthrough. But making a digital human is the hard part.

I KNOW A DIFFERENT COMPANY WAS RESPONSIBLE FOR THE TUPAC HOLOGRAM …
There were several companies involved in that. The one that did the actual visual effects was a company called Digital Domain, which is actually a company that I used to work for. I learned a lot of my trade from working there. I also worked for George Lucas' Industrial Light & Magic, Sony — all the major visual effects studios, learning my craft, until I formed my own business, which is where we are now.

WERE YOU WORKING AT DIGITAL DOMAIN WHEN THEY WERE CREATING THAT HOLOGRAM?
I was, but I was on a different project. I worked on Tron: Legacy there, Transformers, all sorts of stuff. 

IF YOU WANT TO ADD A SONG TO DIO'S SET, HOW DOES THAT WORK?
We use motion capture pretty extensively for this, so what happens is that we have to get back out there on the stage and record the performance again — the facial performance, the whole nine yards. The songs we're going out with this time have already gone through the process of visual effects, so we'd have to go back and do that with all the new songs. It's probably about two months per song, for production.

WOW.
Well, we work at 60 frames per second. Most movies you watch are 24, so it's three times the amount of data that you'd be using for a film. It's consumed a lot of time and a lot of space on our server, that's for sure. [Laughs] Just to put it in perspective, if you're watching the latest Star Wars and you see Princess Leia come on the screen, she's on camera for maybe, I don't know — the average shot in a film is five to ten seconds, and she was seen for maybe five shots. So at 24 for frames per second for five seconds, you're looking at about 120 frames total, which is hardly anything. For us, it's one camera — there's no cuts, so you can't hide anything — each one of our songs is anywhere from about 10,000 frames to 20,000 frames. We gotta make it look good continuously for a very long period of time. That's the biggest challenge.

dio hologram 2019 3, Stephanie Cabral
photograph by Stephanie Cabral

WHICH OF RONNIE'S PHYSICAL DETAILS TOOK THE MOST TIME TO RECREATE?
I'd say we've gone through 80 to 90 iterations just to get the hair simulation to work. For each simulation of 10,000 to 20,000 frames, we're looking at two to three days of just one computer calculating until we have hair that looks like Ronnie's hair.

WHAT IS IT ABOUT HAIR THAT MAKES IT SO DIFFICULT?
Imagine 50,000 strands of hair, or whatever it is, and each one of those has to collide with each other and react to the one around it. For a computer to process that, it takes a lot of time. There's a lot of trial and error. You get a lot of crazy looking simulations that don't look anything like Ronnie's hair. [Laughs] And then you have to completely start over and do it again. So it's time consuming and frustrating at times. One time we'll do an artist that has short hair — that would make my life easier. But that certainly isn't going to happen in the metal world!

AND THAT'S JUST ONE PART OF THE PROCESS.
Right. Before that, we've got all the facial capture, which has to be refined. I've got a team of animators that study Ronnie's performances. They'll use facial capture data, but they still have to tweak and modify it to make it look like Ronnie. On top of that, you've got the lighting. We have to light the model so it actually looks like he's onstage at a rock concert. And before all that even begins, we have to do the digital sculpt.

WHAT'S THAT?
Just like you'd see a traditional sculptor work with clay and sculpt a likeness, we do that all on the computer. I have an artist who will sit there and sculpt with a digital airbrush until Wendy is like, "Yep, that's Ronnie." [Laughs] And we had to go through many iterations of that.

dio hologram rehearsal 2019, Stephanie Cabral
photograph by Stephanie Cabral

HOW MANY PEOPLE WERE WORKING ON THIS AT ANY GIVEN TIME?
Probably around 20 at the height of the production.

HOW DID YOU DECIDE WHICH ERA OF RONNIE WOULD BE USED?  OBVIOUSLY, YOU DIDN'T GO WITH SUPER YOUNG, RAINBOW-ERA RONNIE, BUT YOU DIDN'T USE SUPER-OLD RONNIE, EITHER.
The main thing was that we didn't want him so different from his bandmates, the live musicians, that it seemed odd. So we kinda focused on the year 2000, [just before] the Heaven & Hell era. The original one that we did, three or four years ago, was a younger version. But again, that was just a test. We wanted the one that was going out on tour to be a similar age as his bandmates.

DO YOU SEE THIS TECHNOLOGY AS THE FUTURE OF LIVE ENTERTAINMENT — DECEASED MUSICIANS IN CONCERT AS HOLOGRAMS?
I think we're already seeing it. We knew that something was there, and we've always approached this as… we're not trying to bring somebody back. We're just trying to continue their legacy and provide the entertainment. Do you want to go out and pay tribute to your favorite artist? That's what this is. There're tribute bands, there's videos and documentaries, might as well add a hologram show. The only difference is that this is created from scratch — minus the vocal, obviously.

WERE YOU A DIO FAN BEFORE THIS PROJECT STARTED?
No, but I've come to appreciate Ronnie's music a lot because I've been listening to so much of it over the last three or four years. Once I started listening, it definitely pulled me in. So I feel privileged to be involved in this project. I actually just purchased Simon Wright's drum set that he used on tour with Dio.

YOU'RE A DRUMMER, TOO?
Well, I will be if Simon teaches me!