Cutting Edge
Voice 03
Bridging the Gap between Creators and Technology
Jul 4, 2022
Profile
-
Daniel De La Rosa
Vice President of Post Production Technologies
Technology Development Team
Sony Pictures Entertainment
A collaborative effort to bring key Sony technologies into play
The purpose of “KILIAN’S GAME” was to test several key Sony technologies in an actual production setting, and so it was a collaboration between different organizations within the Sony Group. I served as the executive producer in conjunction with Yoshikazu Takashima, and we partnered with the R&D team in Japan and Sony PCL. There were also a few external partners, the first of which was a production company called Saylor. I’d worked with several of their members on another video test many years before, so I knew that they had the creative vision we were looking for. Saylor also introduced us to a highly regarded visual effects company called FuseFX, who have worked on several movies and television shows including Marvel.
We wanted to test the new technologies, but at the same time, we wanted a bit of a story element, so that people could be entertained. Saylor wrote the script, and one of the things they’re good at is working within parameters. They structured the narrative so that we could shoot in two locations, Los Angeles and Tokyo. The ongoing pandemic meant that we wouldn’t be able to travel to Japan, and the Japanese members wouldn’t be able to visit us, but we wanted an opportunity to test Sony PCL’s new virtual production stage in Japan.
In terms of genre, there were several potential options, including horror, science fiction, and the classical 40s noir detective style that we ended up going with. We evaluated each of these concepts from the standpoints of what would be achievable with a virtual production stage and what would be compelling. Initially, we thought science fiction could be an interesting option, but that’s something quite commonly done, and people typically expect a science fiction film to use some virtual production. We decided on the 40s noir theme because we wanted to make a virtual production so convincing that people would think it was all shot in the same place.
The production process
The opening scene of the film was shot on November 19, 2021. It was done over a road in Malibu using Sony’s new Airpeak drone, which had just been released at the time. For that, we reached out to the business department. They provided us with training on the drone and sent a representative on the day of the shooting. We struggled a little due to the mountains in the area blocking our signals, but we managed to get the shots we needed thanks to the representative’s help. I feel other drones would have not been viable in the same environment or would have required more equipment to manage signals. Afterwards, all the feedback from the pilots was reported to the developers for use to provide new features in a firmware update for the drone.
The second day of shooting took place at a classic Hollywood Mansion called the Paramour Estate. After filming the scene inside the mansion, we scanned the room and built a volumetric model in Unreal Engine (by Epic Games, Inc.), which was then sent over to Sony PCL for the virtual set in Japan. Some of the small props like the bottle were physically shipped over, but all the larger items like the closet had to be recreated in the model. For that reason, we kept the room’s interior relatively simple to avoid creating extra work for the VFX team.
One important aspect of the collaboration was that we had several Sony’s Xperia™ smartphones connecting with cameras on set, capturing everything that we were doing. These feeds were sent back to Japan via a third-party service called MeetMo, allowing the Japanese engineers to monitor the production in real time. Likewise, when they were filming the final scene on the virtual production set in February 2022, we used the same setup to monitor their progress. Our directors Matthew Litwiller and Collin Davis were on hand to give them feedback and suggestions, which they were able to implement immediately. I remember the directors commenting that the virtual set piece looked really good. Everyone on the United States side felt like they were back in the same physical location. On the other hand, I think that having two separate productions on this project resulted in the blending of two different creative genres, American-Noir and Japanese-Noir. This allowed each production team to put their own “creative signature” on each portion while remaining cohesive.
Exposing engineers to the creative process
I think the greatest outcome of this project was that the Sony engineers could get more exposure to the creative process. It’s very difficult for an engineer working on a camera or other piece of technology to get access to an actual production set for a film or television. Through this project, our engineers could see some of the “cheats” that directors use during production firsthand.
For example, during the scene shot in Japan, a lot of the short cuts like the hand pulling open the drawer or the glass breaking were actually shot in the U.S. It makes you think it’s the Japanese actors, when in reality it wasn’t. This is what we call “inserts” and because the directors were also the editors, they knew exactly what kind of inserts we would need. Other tricks included having a microphone just off camera or having an actor stand on a small box to make them look taller for a certain shot. By seeing how the equipment is used in practice, our engineers were able to gain insights that will help them improve Sony products in the future.
In that sense, I think Sony has been doing a really good job lately of bridging the gap between technology and the creative side. Over the past four and a half years since I joined the company, there has been a lot more awareness of getting feedback from creators, and you can clearly see that in some of the improvements on the VENICE 2 digital cinema camera versus the first VENICE. The first VENICE was already a great camera and very well-received, but Sony made the VENICE 2 even better by implementing suggestions and feedback. I’ve been working with cameras for over 30 years, and I have never seen that kind of rapid development from any of the other major manufacturers, so I think Sony is really good at taking feedback
I’m looking forward to building awareness of “KILIAN’S GAME” both within Sony and externally with the rest of the motion picture industry and to the next iteration of this project. Instead of “KILIAN’S GAME 2”, we’d like to do something completely different and hopefully leverage some new technologies in the process. The pandemic isn’t over yet, so I think it’s important that we continue discussions on how Sony can utilize and develop technologies to combine footage shot in different locations. I’m looking forward to seeing what our engineers will come up with and how Sony can contribute to the movie industry in the future.
Message: Daniel De La Rosa
Don’t be afraid of failure and don’t be afraid to push boundaries. We go into these projects knowing that some things might not work, and at the end of the day, we learn just as much from our failures as we learn from our successes.
Technologies such as virtual production are becoming more and more accessible, so I’d like creators to keep experimenting, and I’d like engineers to keep track of what’s happening on the creative side. There’s a whole community of creators who are willing to share their ideas and how they do things online, so it’s important for engineers to monitor those spaces. Sony needs to understand those needs and keep developing technologies that will allow creators to maintain their momentum.
