Millions of moviegoers around the world have experienced the magic of his storytelling — and now this director is trying something completely different.
Jon Favreau has made us laugh with his modern Christmas classic, Elf, dazzled us with big-budget Marvel flicks like Iron Man 1 and 2, and re-imagined Disney’s The Jungle Book completely as a live-action spectacle brought to life through fantastic special effects and performance capture.
Favreau is taking on another Disney classic in a live-action version of the The Lion King, but he’s also overseeing a project with a decidedly different set of challenges. And like his films, it utilizes the cutting-edge of available technology to create a unique experience. The filmmaker has teamed up with tech startup Wevr to create a new virtual reality experience, Gnomes and Goblins, using Epic Games’ Unreal Engine 4 technology.
While everyone will be able to enjoy The Lion King on the big screen in a few years, Gnomes and Goblins is being developed exclusively for HTC’s Vive VR system at this point.
“We made a decision to only port to platforms that have the same level of sophistication,” Favreau told Digital Trends. “So we’re missing out on a larger potential audience, but we’re able to push the edge of what the technology has to offer, which is what’s more interesting to us right now.”
HTC Vive is unique right now in its ability to allow users to move around within a relegated space, which as noted in our review, “is a huge boon to realism.”
“Hopefully the technology will catch up and will become more broadly available,” Favreau said.
“Moore’s Law bears out that over time the technology is just going to get better and better, especially with all the emerging platforms for VR. And our hope is to be there at some point, but we didn’t want to diminish the experience simply because it would be more readily available.”
Vive owners can download a free preview of Gnomes and Goblins now. It’s but a small piece of the overall project, which Favreau said will launch in 2017. The Goblin Forest level invites users into the imagination of Favreau’s enchanted storybook world, where tiny goblins live hidden amidst trees and foliage.
What separates this fantasy world from so many others are its inhabitants, which react directly with the user in ways that traditional Hollywood entertainment simply can’t.
That’s part of the challenge for Favreau and Wevr CEO Neville Spiteri, who’s working with the director on the project as part of a 10-man team that includes Oscar-winner Andy Jones (Avatar) and Jake Rowell, who developed the Call of Duty games at Infinity Ward for 5 years.
“We digitally built these characters from scratch with the AI and animation work being developed hand-in-hand,” Spiteri told Digital Trends. “There are layers of key-frame animation, but most is procedural animation derived by a complex AI system.”
Spiteri said Gnomes and Goblins borrows from both the storytelling of filmmaking and the interactivity of video games.
“It was important to have moments of emotional connection with characters in the story, which was an element from filmmaking and storytelling,” Spiteri said. “But how that got executed is more like a game. Here you’re a participator in the experience. You are the protagonist in the plot and the characters respond to you.”
“If the customers are intrigued by the content, we’re going to see the platforms get better and the hardware get better.”
When you approach a goblin in the VR world, it looks at you and responds to you. You can even pick up objects and hand them to the characters. These types of interactive systems are born in the realm of game design. Still, it’s the hybrid of the two formats that really sells the adventure.
“There was a lot of emphasis on creating digital actors that can deliver performances, but aren’t captured,” Spiteri said. “Jon said he wanted the goblin to be more shy in the beginning and more confident later. We portray that through the eyes and ears of the character, which borrows elements from filmmaking.”
This also opens up replay value for the experience, evident even in this first small preview of what’s to come.
“Part of what’s important for this next wave of VR is rewarding the end user for coming back and re-engaging with the content,” Favreau said. “I’ve noticed that, with a lot of VR experiences, it’s a wonderful thing to experience and maybe I’ll go through it a few times to really understand everything that’s been laid in there, and then I’ll seem to share it with other people who’ve never done it before, and then you leave it on the shelf. You may revisit it later, but unless it’s been updated, you tend to want to look for new experiences.”
As much as the visual design and the task of building believable characters were important in an effort to bring this lucid-dreamlike world to life, Favreau says the team also put a lot of focus on audio to not only help guide people through the experience, but also to make it more immersive.
“You have binaural audio, so you have perspective with the sound effect,” Favreau said. “If you could get it just right and if you have an end user that is comfortable in listening and paying attention to the directionality of the sound, it could be very helpful. I find that sometimes people who aren’t used to VR or are used to just watching movies or television … will take a far more passive role and miss a lot of the action because they’re being steered by the sound design and it’s sometimes lost on them.”
Even though virtual reality is essentially a solitary experience right now, Favreau says that, since many people experience it with friends at home or with groups of participants at public events, it takes time to fully appreciate the depth of a good VR experience.
“VR tends to be a community experience where one person will be trying the experience and he or she will have other people around them watching on a screen and talking to them and distracting them,” Favreau said. “So in a completely immersive situation, it gives you a whole other set of tools by which to help guide the focus of the user. But when people are overwhelmed by it because they’re using it for the first time or they’re using it in groups, those subtleties are often lost. With Gnomes and Goblins, where a tremendous amount of effort was put into the sound design, we find that in the first time through the experience many people miss a lot of what we had made in there.”
Since Gnomes and Goblins is still in active development and there’s still plenty of time before its ultimate full release, Favreau is hopeful that more platforms will evolve to allow for the kind of experience his team is creating with Gnomes and Goblins.
With advances in virtual reality across multiple platforms, including Facebook’s upcoming Oculus Touch, PlayStation VR, and the launch of Microsoft’s Windows 10 mixed reality headsets in 2017, Gnomes and Goblins may well be positioned to expand its footprint down the road.
“I think all of that is going to catch up as people start to become more comfortable with the technology,” Favreau said. “You have really strong players in the game right now who are great at developing these technologies for their platforms. And if the customers are intrigued by the content, we’re going to see the platforms get better and the hardware get better.
“Ultimately, we’re all following the customers and what their needs are. It’s incumbent upon designers to develop killer apps that test the limits of the technology, and thereby create a demand for the technology to continue to innovate. That’s the exciting cycle we’re entering into now.”
You can get a first look at Gnomes and Goblins at the Steam store now.
Updated 12-6-2016: This post was edited to clarify it is Jake Rowell, not Powell, who worked at Infinity Ward, and that he worked there for 5 years.
