Initiatives
"Empower creators' creativity"
Sony Technology Day Report Vol.3
Dec 6, 2019
At Sony Technology Day, which was held on September 18, 2019, Sony exhibited its technologies on three themes: "Connect," "Empower," and "Exceed." In this report, we introduce the theme of "Empower" and the technology exhibits that empower the imagination of creators through extension and fusion technologies.
Real-time ray tracing technology
that pursues reality by freely manipulating light
We introduced the ray tracing technology that generates photorealistic CG images in real time by physically and correctly simulating rays such as reflections and refractions.
Since ray tracing requires an enormous amount of computation to calculate the scattering caused by the reflection of light emitted from a light source, we developed a system that runs massive GPUs in parallel to realize real-time processing. Deep learning is also used to infer the diffusivity and reflection of light on different materials, making it possible to render material colors and “textures” in real time. By using this technology, it is possible to produce video games that look like live-action movies, and it is expected that the quality of CG production will be improved with reduced costs at a film production site.
6DoF video technology that makes a captured person appear
in a virtual 3D space
This exhibit showed a technical demonstration that creates video content with a natural sense of distance and immersion by synthesizing live-action images into a three-dimensional virtual space.
Video content that provides a natural sense of distance is achieved by changing the perspective and relative positions of objects in conjunction with the natural movement of a person within the video. This technology uses the 6DoF (Degree of Freedom) viewpoint. We have also developed a unique technology based on deep learning to extract a human image without using a special background for chroma key synthesis. In the future, we hope we can bring new visual entertainment to the world with the aim of providing a more immersive and natural visual experience.
Lightweight motion capture technology for
easily capturing human movements
This demonstration showcased a unique motion capture technology that captures the entire body movement by simply using six small, lightweight sensors attached on the body, showing how the actual movement can be easily reflected in virtual space.
With this technology, it is possible to estimate the positions and orientations of all joints of the human body from the sensor data only by attaching sensors to six places: the head, waist, both hands, and both feet. A strong point of this technology is that it can estimate the intermediate joints, such as elbows and knees, where no sensors are attached. The demonstration especially showed that a character synchronized with the user's actions was synthesized into a video with a simple setup.
The Volumetric Capture technology to capture the entire space
In this exhibit, we introduced a technology called "Volumetric Capture," which captures real persons and places and converts into three-dimensional digital data to reproduce them in high image quality.
The demonstration was performed in two ways, focusing on a human and a location, separately. In the "human" demonstration, two dancers were captured in the Volumetric Capture studio, and their 3D images were shown in VR/AR applications. In the demonstration of "location," a real set used in a movie was captured and formed into a 3D digital image.
The Volumetric Capture technology consists of three parts: capturing, displaying, and transferring. By combining technologies in all these three parts, Sony can realize value for the first time, and that is possible using Sony's diverse technologies.
