Marketing Best Practices

Toe-Tapping Tech: The World’s First Virtual Reality Musical

By Andrew Littlefield March 7, 2017

There’s a moment of truth in virtual reality movies. A split second where the viewer decides whether they’re impressed or underwhelmed.

But it’s probably not the moment you think it is.

No, it’s not when the viewer first puts on the headset. Or when they realize they can look around. Or when they inevitably reach out their hands, attempting to touch what’s on the screen in front of them. That’s all interesting, but it’s not the real moment of truth.

The moment of truth is when they pull off the headset.

For about two seconds, the viewer has to reorient themselves to reality. Their hair is a mess, tangled in the head strap. Their eyes squint in the light, and there’s the faintest twitch of their head as their brain accepts the fact that it didn’t see what it just saw.

It’s a brutally honest moment. Everything you need to know about someone’s reaction to a VR experience can be gleaned in those two seconds; it either hit them square in the emotions, or it didn’t.

Like any new technology that comes along, virtual reality is going through some growing pains—there’s a rush to start using it, no matter the circumstances. This inevitably leads to a lot of poor-choices (like synthesizers in the 80s). The rush to produce something—anything—using virtual reality creates a lot of low-quality junk. Producers pick VR not because it’s the most effective medium to tell a story, but because it’s the latest shiny toy.

In the short term, this may grab a few eyeballs as people flock to test new technology. But it also can hurt the long-term viability of new media. If your first experience with VR is underwhelming, how likely are you to come back?

For filmmakers like Joshua Howard, that’s part of what makes VR such an enticing challenge. Howard is a virtual reality filmmaker, screenwriter, and producer, who is looking to push the medium beyond the status quo—not just the tech, but how that tech is used.

It’s easy to fall into the virtual reality trap, where the powerful technology serves as nothing more than a gimmick, rehashing the same boring techniques that get done over and over again. But Howard doesn’t just want audiences to be immersed in an experience. He’s looking for something more.

He wants to make them sing.

Don’t Say It, Sing It

Joshua Howard is a seasoned storyteller. His résumé includes award-winning photojournalism and cinematography projects with clients like PBS, Apple, and National Geographic. But Howard, who’s earning a second bachelor’s degree at the Savannah College of Art and Design (SCAD), didn’t anticipate his career taking him into the virtual realm.

“This was a total career change,” he says. “My background is in photojournalism and cinematography, but it’s always been about the storytelling for me.”

SCAD wanted to dive into the VR world, but in a nonconventional way that would set them apart from other virtual reality filmmakers. They approached Howard to write a screenplay, avoiding being overly prescriptive in favor of trusting his artistic instincts to take it in a creative direction.
“When we set about to create this VR narrative, there were so many genres to choose from and we looked at all of them,” says Howard. “But so many of those genres lend themselves to VR almost too easily.”

It’s not hard to find horror films in the VR space. Scaring people with a monster that flashes right in front of their face isn’t too terribly difficult. And placing people in unusual sensory experience (a roller coaster or tall building) isn’t much of a challenge in VR either. But Howard and his team wanted to push themselves—and the platform—in a direction it had never been taken.


“We ended up landing on a romantic, musical, comedy,” he says. “Sort of an homage to 1920s Irving Berlin and classic Hollywood.”

Thus, Say It With Music was born.

They picked a pretty good time to release a musical, with six-time Oscar winner La La Land bringing a resurgence of interest to the classic genre. And the SCAD production is the first musical to ever be taken into the VR medium. That alone would be enough to set it apart. But to get the reaction they really wanted—toe-tapping and ear-to-ear smiles—they would need to do more than sing in front of a fancy camera.

The success of this experimental VR storytelling project would be in the tiniest of details.

Going Old School With New School

It’s easy enough to get a 360° camera and make something in VR. These days you could do it for less than $300. Order a camera off Amazon, and two days later you can plop it on a tripod and start filming.

But that’s not really taking full advantage of the benefits—or considering the weaknesses—of virtual reality.

“Right now, VR has the cool factor going for it,” says Michael Chaney, a professor of film and television at SCAD. “But once we get past that, we need to ask ‘How do I make viewers a part of the story?’”

Tech geeks (myself included) tend to focus on the immersive aspect of VR and view that strictly as a benefit, but it also limits directors a bit. Traditional filmmakers can use hard cuts, close-ups, interesting framing, and a whole host of other camera techniques to tell their story better. Even something as simple as a back-and-forth conversation can be completely changed by how the director chooses to shoot it.

VR takes a lot of those tools out of the director’s hands. Sharp cuts and zooms are likely to make the viewer a bit nauseous—not the recipe for an enjoyable experience. Instead, VR filmmakers must rethink how they frame and shoot action to direct the viewer’s attention naturally.

“Transitions between one place and other is something we’re still trying to figure out,” says Chaney. “Another challenge is acting techniques. When you’re up close to somebody (like in VR), a lot of traditional acting techniques don’t work well.”

Despite the challenges, Chaney urges those who wish to dive into VR to always remember the basics of storytelling: structure, character, and theme. “It’s a lot like cooking. I grew up in Mississippi, so everything I cook has three basic ingredients: onions, celery and bell pepper,” he explains. “No matter what I make, I know I have to start with those. It’s the same with storytelling, doesn’t matter if it’s comedy, drama, or a VR experience.”

Melodies Mellow, Played on a Cello

Musicals, more than any other genre, present a whole lot of logistical challenges that you don’t otherwise run into with traditional movie making.

You need composers to write and arrange the music. You need choreographers. You need actors who can sing (and dance).

Throw virtual reality into that mix and things get even more complicated.

That’s what made Say It With Music such a unique challenge. Making a musical in VR is more like writing for the stage than the screen. “In the 360 space, you can’t really hide anything,” says Howard. “As a director and writer for VR, you have to be okay with the decisions the viewer makes. If they decide to just watch the person behind the bar, that person has to be the most important part of the film (for them).”

Unlike a typical film that utilizes extras as background filler behind the main shot, in VR the people in the background could easily move to the foreground, depending on what the viewer decides to set their gaze upon. While you can attempt to guide their view through action and movement, being overly prescriptive with this defeats the purpose of the virtual reality experience—viewers are supposed to have curious eyes, otherwise it might as well be on a traditional screen. If extras are static characters, the realism of the experience is suddenly thrown off balance.

Because of this, Howard and his team decided to write individual story arcs for every single person in the film—all 36 of them. “We never really called them ‘extras,’ they were really more of a supporting cast,” he says. “The replayability factor of this is interesting, because while the film may only last four minutes, if you want to watch every story arc and uncover all the details, it might be a 45-minute experience.”

That’s where the real difference in their work comes in: immersing viewers into a believable world that happens to also be a musical. That attention to detail is vital to the success of the project. The slightest hiccup can break the viewer out of that immersion and kill the experience. But by focusing on the details, Say It With Music achieves that magical “back-to-reality” moment seconds after removing the headset.

“If they take off that headset and they immediately have our songs stuck in their head and a smile on their face, we’ll know we’ve succeeded.”

Say It With a Beautiful Song

Say It With Music is currently making the festival rounds before being released to the public, but if early feedback is any indication, they’ve struck a new chord with viewers. “We’ve had everyone from a five-year old girl to an 87-year old woman watch it, and the response was great,” says Howard. “They had a smile on their face and they were tapping their feet.”

Getting that type of response is far from automatic, VR or not. The response Howard is looking for requires a suspension of disbelief. This is especially true for a genre as stylized as musicals—when everyone in the room starts singing a song and dancing together, the viewer must be invested enough to roll with it and enjoy the experience. When it’s done well, the experience is nearly spiritual.

Take it from Roger Ebert, who once said that “every once in awhile I have what I think of as an out-of-the-body experience at a movie… my imagination has forgotten it is actually present in a movie theater and thinks it’s up there on the screen.”

Virtual reality presents an opportunity to do this in incredible new ways, but only if the filmmaker takes care and precision with details. Physically placing someone within a scene only gets you so far. To truly give them an out-of-body experience, they need to forget about the cumbersome headset strapped to their face and just enjoy the song and dance.

And there’s only one way to do that: tell a great story.

  • Tyler Moore

    So is this a 360 video or actually stereoscopic? I really can’t get into 360 videos, the lack of depth perception is just killer for immersion.

  • The rig doesn’t look stereoscopic, it’s four cameras for each main area with overlap.

  • Is there a link where we can see the actual video of this production? Or, at least some 360 immersive clips?

  • Hey Brian,

    Not quite yet. The film is making the rounds in the festival circuit at the moment and won’t be released until after that. They’re looking at a few options for distribution right now. Josh mentioned YouTube as well as some VR specific streaming sites.