Last week we turned our spotlight on the Captain America Experience Digital App that Marvel and Walt Disney Studios launched. This Augmented Reality application for Smartphones turns a flat 2-D movie poster into a totally mind-blowing 3-Dimensional image, with the option to take a picture and share it on your favorite social network!
We caught up with Nels Israelson who photographed the cast of Captain America: The Winter Soldier (out in theaters today) and helped spearhead this amazing App. Click ahead for our interview in which he gives us some behind the scene details about the process and technology that is truly a super-hero endeavor.
In Captain America: The Winter Soldier, Steve Rogers struggles to embrace his role in the modern world and battles a new threat from old history: the Soviet agent known as the Winter Soldier.
Nels Israelson is a Los Angeles based professional photographer who has worked on some of the biggest marketing campaigns for some of Hollywood’s biggest films and TV shows. Some of his credits include; Divergent, The Hobbit, Spider-Man 2, X-Men, Game of Thrones, Mountain Men, Sherlock Holmes: Game of Shadows, Pirates of the Caribbean: At World’s End, John Carter (of Mars), Daredevil, Green Lantern, RED, Fantastic Four and many more. He has also worked with various collaborators on innovative 3D capture processes for over 20 years. Some of his first 3D movie posters were produced for Disney: the Nightmare Before Christmas (1993) and Mighty Ducks 2 (1994).
Beyond the Marquee: Nels, tell us how this new Augmented Reality technology is different than AR programs we’ve seen in the past.
Nels Israelson: Well, the Captain America Experience (CAE) app marks a key breakthrough in Augmented Reality tech: its unprecedented photorealism sets the CAE apart. We have deployed this process with Disney to essentially break the “fourth wall” of the movie poster, bringing the spectacle and promise of modern high-tech movie-making out of the theater and, literally, directly into the hands of the audience. In designing this experience for the mobile platform we also are tapping into these devices’ capabilities as both social networking portal and media channel.
BTM: Sounds amazing, so can you give us an idea on how the technology works?
Nels: Technically, the AR app functions by “triggering” the AR effect when your device’s camera recognizes a specific target or “marker” image (the movie’s posters). Subsequently, the app continues “tracking” the marker so that the 3D image appears to lock on & replace your view of the poster with an enhanced, dimensional, photorealistic scene.
BTM: I loved the fact that I could hold my phone above and below the image once it triggered out of the poster and explore around it. It reminded me of the “bullet-time” effect from the Matrix in many ways. It was amazing to check out various angles of Captain America as if I was looking at a real person in costume standing right in front of me, while trying to avoid shards of digital broken glass that was shattering out around it.
Nels: Yes, it is pretty incredible and in a way you could call this a kind of “real-time Photoshopping” that provides the illusion that the film’s characters are present in front of your device for you to experience and take your own unique photos with. Of course, seeing only these user-generated photos is only a hint of the actual AR effect– it does not compare with experiencing the dimensional presence itself. (Especially when your device shows you Cap, 90 ft tall, towering over the Sunset Strip!)
BTM: So tell us about the company who came up with this technology.
Nels: The software team I collaborated with on the CAE is called Digicave. They are innovative leaders in the development of Augmented Reality technology and have been working at the forefront of 3D AR technology for over 4 years. CAE is our first joint project, which we began in May 2013. Because it has been under an NDA (non-disclosure agreement) since then we don’t have anything online regarding the work or the specifics of the technology. And the ongoing work we’re doing to further develop the process continues to be NDA, so we remain under-the-radar. But the Digicave site and their LinkedIn page has a bit of mission-statement.
BTM: So you provided us with an image of the stage during the actual shoot for this Captain America Experience App, can you elaborate on what we’re looking at here in the picture?
Nels: Sure can, this an image of the actual CAE capture setup from our shoot in Cleveland. It is called an “array” of 63 cameras, each 36 megapixels, all firing at once — so the combined data capture for each “frame” we shot of the cast members was 2268 megapixels, or 2.2 gigapixels.
BTM: That’s a whole lot of data, how does it all come together?
Nels: The 63 individual captures are processed through proprietary software into a single “model” that is similar to a CG (“computer generated”) image, but we call it a “computer mediated” image. This process creates a freely rotatable model. Unlike other forms of 3D model capture, our process retains the exact pose and lighting of the captured moment. This is what makes our particular AR so uniquely photo-realistic.
BTM: Nels, Augemented Reality has been around for some time now, but hasn’t seem to really caught on… it just seems to pop up as an extra fun promotional marketing gimmick. Do you think now with the exposure of this technology on such a huge film such as Captain America: The Winter Soldier, that we’ll see a boost in the AR usage out in the world?
Nels: I think we’re in the very early days of this kind of interactive technology, and it will quickly get even better. As people continue to adopt the mobile platform as their primary access for internet and social media, we’ll only see more cool content that can uniquely play to this rapidly advancing medium: it has a camera on one side, a screen on the other, and instant connectivity to all our friends (and the world!) in between. We’ve gotten used to routinely sharing images of our experiences from our smart-phones, but what this app does for the first time is make your device hallucinate. Not only is this a crazy spectacle to experience, you feel compelled to immediately share it.
BTM: This week the companion piece to Steve Rogers as Captain America bursting out of the 2-D poster has been revealed, and it’s The Winter Soldier exploding out and catching Cap’s Shield. It is bad ass! Tell us about what we can expect from that image.
Nels: Yeah, I agree, the Winter Soldier is bad-ass! The shield-catch is a major moment in the story, and we’re honored to have had an opportunity to bring it to life in a new way for the film’s audience. For that AR scene, we separately shot both the front & the back of the shield with our camera array so as to build a full 360 view of it. You can walk your device right up underneath to check out the straps.
BTM: Any last thoughts on the Captain America Experience App as the film opens Nationwide today?
Nels: It’s all a part of transmitting the excitement of the film out into the world as an integrated aspect of the film’s campaign. I’m grateful that Disney has been willing to take a risk with this technology for the Captain America: The Winter Soldier marketing: in a way we’re literally breaking right through the surface of the 2D printed piece, and bringing the film poster into the 21st century.
BTM: Thank you Nels, really appreciate your time.
Nels: You bet, appreciate the chance to chat with you as well.
(CLICK HERE to Check Out the Pics of “The Winter Soldier” 3-D images from the CAPTAIN AMERICA EXPERIENCE)
Download the Captain America Experience App: