How Japanese rock star Miyavi performs in a world without live music

Miyavi Virtual 3.0 will be available to buy and stream, it is a live performance mixing drone footage with dazzling digital art. It turns out that recordings for glitzy live-streaming productions can involve a lot of waiting around well past midnight. But that is just how it goes in 2020. As the covid pandemic continues to rob creators of the ability to play in front of live audiences or even record music videos in traditional studios, Miyavi and his creative teams are resorting to technology to keep performing in front of fans.

Born Takamasa Ishihara in Osaka in 1981, Miyavi is best known for his fast-paced, catchy electro-rock music and frenetic guitar playing. Miyavi started out in the visual kei band Dué le Quartz before embarking on a solo career that eventually began to encompass modeling and acting, including roles in Hollywood movies like “Unbroken” and “Kong: Skull Island”. In other words, Miyavi tends to have a full calendar, and 2020 was supposed to have been no different.

Miyavi normally bases himself in Los Angeles, and was booked for some movies, TV projects, and a Gucci campaign. Then, Miyavi’s latest solo album “Holy Nights” was set for an April 2020 release with a tour to follow. But by the beginning of March 2020, Miyavi found himself back in Tokyo, and it was clear that plans would have to change. For creators, they can not do live performances now.

So it is really a crucial time to find a new way, a new normal.

To start with, Miyavi’s need to work in both Japan and United States complicates matters. Each country has a different situation. Usually Miyavi has his team in Japan in Tokyo, and a team in America, so they all talk. But the situations with covid they are in are all different. So they heard it is not the time to shoot a music video. But they did not stop, because in Japan at that time record stores were not closed.

But it soon became evident that plans to shoot two music videos in United States were not going to work out. In United States, the emergency declaration happened, and as soon as Miyavi found out that he cannot fly out to America, he has got to switch to more virtual creation. That is why Miyavi started making the music video for “Holy Nights” with his animators, so that creates a world without having him.

He dropped the anime-influenced “Holy Nights” video, developed by his American creative team, on YouTube on May 10th 2020, proclaiming it to be the beginning of “Miyavi Virtual”. But an anime music video does not capture a real performance. For that, a new technological approach would be required. This is a technique that employs several cameras at once to capture a 3D model that can be used in CGI renderings.

It allows for virtual video that is based on an actual performance, and it is easier to do safely than a traditional shoot.

The only physical shooting Miyavi did was in a volumetric capture studio in Japan. It was a minimal team in Japan, pretty much just Miyavi recording 3D models of his movements. This technology is between virtual reality (VR), augmented reality (AR), and reality, so it was really fascinating. It was a really interesting experience, and it is the future.

The key for the future is how real we can feel in a virtual world. The capture studio has dozens of cameras pointed at Miyavi recording video that is then processed into frame-by-frame 3D models played back in the game engine. The amount of data is huge, and sending that data from Tokyo to Los Angeles is a big deal as well. The result looks like a video game starring Miyavi.

The effects were all added later based on his performance. The next step was figuring out how to perform live online. At first he turned to streaming on platforms like Twitch, YouTube, Weibo, and Line, setting up a studio in his home and performing a “Virtual Live” concert standing at his desk. Unsurprisingly, the results were not quite as slick as the “Need for Speed” video.

It is quite challenging, so Miyavi realized that he is not a professional at this kind of thing.

But Miyavi just did not want to stop, because he did not want to give up, just wanted to find a way to move forward. And Miyavi still thinks that was the right move even if it was not perfect, at least he was able to be connected with his fans and share time with them with his creation. Miyavi then wanted to try combining high-tech experimentation with live broadcasting.

The broadcasting was done from a studio in Tokyo with drones. That was the biggest priority for Miyavi to try. And that is the approach that Miyavi is trying to do in the real world. Miyavi Virtual Live Level 2.0 was a performance with a live drummer on a socially distanced stage with no audience except for the fleet of drones buzzing around the band and capturing its performance.

You can do anything with volumetric capture, but it takes time for the production, and Miyavi wanted to do broadcasting using his real body. It is really important to experiment. Much of the video capture is also by drone, and the mirrors and lights surrounding Miyavi make it look like he is performing in the middle of a spectacular void. While the original plan was for a live performance, the decision to switch to a pre-recording is due to the intention of creating and prioritizing artwork that has high video quality, amazing light design etc.

Even before the pandemic, virtual concerts were getting increasingly popular, with Fortnite at the vanguard.

As far as the actual experience of Fortnite in-game concerts goes, however, there is room for improvement. It was 90 percent virtual, with the music made by human beings. This experimentation is the direct result of a desperate period in human history. The global situation will prompt people to live and work in new ways even after things improve. The world is going to be different, no matter what.

Even if we get back to normal, our lifestyle will be different, and the way we create will be different. Because now we know what we can do. This situation kind of pushed us forward.