Collaboration
Unleashing the Power of IP to Reimagine Experiences
[Part 02] In pursuit of the magic moment
Jun 28, 2024
At CES 2024, Sony revealed an innovative approach to immersive storytelling, made possible through close collaboration among several teams. We spoke with Johan Hansson and Ryo Yokoyama to explore the potential of content IP, using the CES exhibition as an example.
-
Johan Hansson
Sony Europe B.V.
-
Ryo Yokoyama
Sony Corporation
Holding IP and technology opens the door to innovation
──What is your current role at Sony?
Hansson：I head up Advanced Content Creation - LBE Section, which focuses on developing interactive experiences using Sony's technology, particularly leveraging game engines.
We are based in Lund, Sweden and we collaborate with game developers and designers to create large-scale interactive experiences. We are part of the Technology Infrastructure Center (TIC), Sony's internal research and development division.
Yokoyama：I'm a researcher at Sony Corporation's Technology Development Laboratories. In my role, I lead research on haptics technology for location-based entertainment (LBE), a field I've been involved in for 10 years.
Breaking Boundaries: Sony's immersive IP exhibit at CES2024
──What inspired the immersive exhibit at CES?
Hansson：For CES, we wanted to showcase an experience that falls between a movie and a game, taking advantage of the LBE technology that leverages physical space for a distinct experience.
Our team is developing an LBE-SDK for game engines which we aim to share with creative teams within Sony. The LBE-SDK allows us to more easily create large-scale interactive experiences, often to showcase IP from Sony Pictures Entertainment (SPE) or Sony Music Entertainment (SME).
To create a unique experience using our content IP, we have carefully selected technologies such as "haptics," a technology that creates a sense of touch; "markerless motion tracking," a video-based motion capture system; and "mocopi," a mobile motion capture system. These technologies, particularly the Haptic Floor, which vibrates the floor in response to images and sound, and markerless motion capture, offer capabilities that go beyond traditional systems, enabling us to capture the movements of many individuals in a large space and provide an immersive haptic feedback experience.
──What role did your team play in this project?
Hansson：We utilized our LBE-SDK to create the technology platform and extended our existing framework with technologies such as haptics to ensure that all technologies were harmonized within the same system. We then implemented interactivity and added visuals delivered by Pixomondo and sounds effects from Ryo's team and PCL to deliver the final experience showcased at CES. In addition to this, we were also responsible for space design and the incorporation of technology, considering both technical and experiential aspects.
Ghostbusters TM © 2023 Columbia Pictures Industries, Inc. All rights Reserved.
Bridging Creativity and Technology through Collaboration
──Sony's theme at CES was "Powering Creativity with Technology." How did your teams approach showcasing this concept?
Hansson： Our objective was not only to showcase Ghostbusters IP but also to highlight how our technologies could empower future creatives by using technologies. We aimed to create an emotional moment, "Kando," by integrating technologies into stories to engage users. We also focused on introducing the technology to users effectively.
In the past, we had observed that users can be hesitant to interact with technology if they're unfamiliar with it. To address this, we created a small introductory space before the main exhibit where users could familiarize themselves with the haptics technology. This approach encouraged more active participation from users, enhancing their overall experience.
Ghostbusters TM © 2023 Columbia Pictures Industries, Inc. All rights Reserved.
──How did the technology teams help bring this idea to life?
Yokoyama：The base frame was primarily designed by Johan's team in Lund and Guy Wilday's team in L.A. Our role was to determine if the haptics technology could be applied to this concept.
We used two kinds of floor-type haptic technologies, one is Haptic Floor and the other is Active Slate with interactivity. Although they serve a similar purpose, their range of expression differs. We had to decide the layout for each area, which was challenging as we had never used both the Haptic Floor and the Active Slate simultaneously before.
Hansson：That's true. For me and my team, one of the major technical challenges was integrating multiple technologies seamlessly. Each technology is usually developed to function independently. Therefore, ensuring that all these technologies can work together harmoniously and leverage each other's strengths was a significant challenge.
──What did this exhibition demonstrate about the future of immersive IP?
Hansson：The exhibit demonstrates how we can take conventional media, such as the proof-of-concept Ghostbusters short film and deliver a new type of experience in a different environment and space. It's neither a game nor a movie; it's something unique. This approach allows us to expand the IP, similar to how a movie can be turned into a game or vice versa. We can create physical experiences based on music, movies, or games to create stronger a relationship with communities. The interactive aspect allows fans to be immersed in the world of their favorite IP like never before.
The CES demo specifically showcased our vision with the Ghostbusters short film, but the concept can be applied to any Sony IP, including games, music or other movies. The goal is to provide fans with new experiences and connections that conventional media cannot offer.
Ghostbusters TM © 2023 Columbia Pictures Industries, Inc. All rights Reserved.
──After the success at CES, are there any new projects or areas you're eager to explore?
Yokoyama：Following the successful collaboration and communication within the group for the CES exhibition, I came to realize how much we can achieve greater things together.
To further facilitate collaboration, we are providing the Lund team with the Active Slate and the Haptic Floor technologies. This will mirror the environment of the Japan team, facilitating continuous co-development.
Hansson：For me, the biggest challenge lies in harmonizing different technologies to create a cohesive experience. It's not just about having them run simultaneously; it's about combining them to innovate in a meaningful way. For example, in our exhibit, we merged body tracking with floor sensors. This allowed users to interact with virtual elements like marshmallow goo, enhancing immersion.
When technology seamlessly merges with the experience, it feels like magic. And we want to produce more of that magic.
This article is the second part of a two-part series delving into Sony Group's content IP business. To learn more about Sony is leveraging IP to drive innovation from Guy Wilday, VP, Interactive Technology, Sony Pictures Entertainment, check out part one of this blog series here.
