People
What is uniquely Sony about our AI x Robotics?
Mar 1, 2019
It has been about 20 years since Sony launched R&D in the field of AI x Robotics. With the rapid changes in industry and market trends, what kinds of innovation have there been in Sony’s R&D? What has remained constant over the years? What lies ahead for AI x Robotics at Sony? We sat down for a talk with three researchers who have been at the forefront of Sony’s R&D to discover the answers to these questions.
Profile
-
Jun Yokono
Vision System Technology Development Department,
Fundamental Technology Research and Development Division 1,
R&D Center,
Sony Corporation -
Kenichiro Nagasaka
Motion Control Technology Development Department,
Fundamental Technology Research and Development Division 1,
R&D Center,
Sony Corporation -
Yoshiyuki Kobayashi
AI Core Technology Development Department,
Fundamental Technology Research and Development Division 1,
R&D Center,
Sony Corporation
Sony’s history in robotics begins with the D21 Laboratory
──To begin with, could you tell us how the R&D structure for AI x Robotics at Sony came to be what it is today? How did it develop over time?
Jun Yokono：If we go back to the beginning, one of the creators of Sony’s AI x Robotics is Masahiro Fujita, who worked on the original AIBO. The first AIBO debuted in 1999, and Toshitada Doi, who was also involved in that project, was the director of the D21 Laboratory (D21). Fujita-san worked under Doi-san, and having worked on GPS and other technologies, Fujita-san was knowledgeable in neural networks, as well as hardware and software, and he was appointed to lead AIBO development, where he could put his full expertise into action.
Kenichiro Nagasaka：The lab was restructured every two or three years. After the D21 Laboratory, in the early 2000s, there was the Digital Creatures Laboratory (DCL), which, as the name suggests, focused on creating digital versions of living creatures. Later, the Entertainment Robotics Company was established as an organization focused on the robotics business, and the lab was reorganized as Sony Intelligence Dynamics Laboratories, Inc. Essentially, the lab was divided into two organizations: one to handle commercialization of robotics, and one to handle R&D on autonomous intelligence systems.
Yoshiyuki Kobayashi：After we discontinued our robotics business, these branches were reconsolidated under the Information Technologies Lab. Then through the several organizational changes, the organization underwent a transition to today’s R&D Center’s Fundamental Technology Research and Development Division 1. All the while, R&D into AI x Robotics continued uninterrupted despite these changes.
──What is most important to Sony in its robotics R&D?
Nagasaka：Up to the mid-2000s, our robotics focused on entertainment robots, which meant developing robots with an emphasis on the emotional value provided by tapping into the emotions of the user. When we pulled out of this business in the mid-2000s, it spurred a great deal of reflection. We saw that people sometimes quickly grew bored of the technology as it existed at the time, and that, to hold their interest, robots would need human-like intellectual capability. Since this wouldn’t be available immediately though, we decided to focus on the physical assistance that robots could deliver, adding that sort of usefulness to our robots to make our business more sustainable. It was at that point that we began research into robotics that could provide people with physical assistance.
Today, we believe it’s possible to create something new in robotics by a high-level integration of these physical assistance capabilities with intellectual processing capabilities.
Yokono：When the first AIBO came out, the idea was to create a robot that would make people feel happy—it didn’t have to be useful. But, as Nagasaka-san said, people sometimes quickly got bored with it and moved on. All of the original AIBO’s behavior was programmed, and we thought that we could keep people from getting bored by augmenting the variations of behavior patterns. But, as it turned out, people lost interest more quickly than we had anticipated, in some cases … It was evident that we needed to address the intellectual aspects of the robot. We came to believe that we would need a new type of robotics, a kind that evolves and enables the robot to change its own behavior autonomously as it adapts to its environment.
A few years after the AIBO launch, around 2004, I was working at the Intelligence Dynamics Lab, researching how to create a new type of intelligence that would keep people from getting bored with robotics. This was the predecessor to today’s research on deep learning. We pioneered a machine learning algorithm that used some 500 CPUs to perform learning using time series and neural networks with hierarchical structures.
Potential in medicine and other sectors
──Technological trends change with the times, and I imagine the role that R&D plays has changed accordingly. But if there is an aspect of the uniquely Sony R&D that has remained constant despite these changes, what would it be?
Kobayashi：In a sense, Sony’s R&D is fairly bottom-up. While the big picture of the organization changes with the changing times, individual researchers stick with their own research themes and pursue their work with a dogged determination to create results. So, personally, I feel like I’ve always just done what I wanted to do.
Nagasaka：That’s true. The reality is that research projects which researchers believe to be important have been continued without pause despite those changes.
──Will researchers be expected to collaborate with people in other fields, while at the same time digging deeper into their own areas of expertise?
Yokono：Yes, I think so. For example, I’m involved in image recognition, and this field requires knowledge of neuroscience and developmental psychology, as well. While the methods of execution may differ, it is important, I think, to strip the human being naked, so to speak… to study the processing mechanisms that we are made up of, since there is no other being at this point in time as intelligent as humans.
Nagasaka：It’s difficult to constitute a business with robotics technology alone. In certain aspects, robotics technology only gains value when it can be integrated into other business areas. In Sony’s case, our business areas include, for example, the entertainment, audio, and visual categories, but when it comes to developing business in other areas, we cannot do it alone. The medical domain is a good example of this.
Sony joined forces with Olympus to form Sony Olympus Medical Solutions Inc., which has developed a neurosurgical microscope system. A number of different projects have come out of this experience. In the area of laparoscopic surgery, for example, rather than cutting open the abdomen, doctors can now open an extremely small hole, insert a long, thin special forceps and a long, thin endoscope, and skillfully manipulate just these two instruments to perform the operation. In the past, a doctor called a scopist was required to hold the endoscope throughout the entire operation. Sony seeks to improve conditions in the operating room by making it possible for robots to provide support by holding the endoscope.
First, train people to use tools
Kobayashi：I work on machine learning technology. Machine learning can, in all honesty, be used with anything. With Sony involved in so many different businesses from movies to music to finance, I listen to what my colleagues working in these areas have to say and offer suggestions like, “this type of machine learning can be used for those tasks.” In certain cases, we work with the division to conduct testing and often, when we find that what we’re testing will in fact work, developers will be assigned at the business group to help them move forward with commercialization. Working with the user to discover practical applications is part of what we do.
In the past, this work was limited in scope to within Sony, but last year, we started working with companies other than Sony as well by making our deep learning framework available in an open source format and distributing the development tools free of charge. In fact, meeting with people outside the company and listening to what they have to say, we have seen a range of concrete needs that weren’t obvious inside Sony. It’s very interesting, and I expect this will speed up our research, as well.
Yokono：What is great about Kobayashi-san is that, whether it is inside or outside of Sony, he first trains people to use the tools. By so doing, he expedites the process of getting the technology into wider use. When I was working on image recognition and it never crossed my mind that I could employ that kind of strategy. This is indeed a great way to get our technology used by more and more people.
Kobayashi：I think there are two types of people who are focused on creating a new world now. One type is those who create the best technology in the world. And I think if we were to categorize ourselves, we fall into this first type. We provide the image recognition technology that delivers the best performance and offer technology to smoothly control robots. The second type is people who are capable of combining various technologies that exist in the world to create solutions quickly. I think a major role that R&D plays is to provide this second group of people with technology that is easy for them to use in a precise manner.
Potential to collectively solve various problems facing the world
──Finally, in thinking about AI x Robotics going forward, what types of skills does Sony need employees in this area to have?
Kobayashi：Personally, as I mentioned earlier, I think we need two types of players. First, we need employees working on technology development to create the best technology in the world. We will be left behind quickly if we do not deliver the best in the world. We need people with the ability to focus on this, develop technology with this goal in mind, and be prepared to fight for it. The second type is the artist—a type of person who is able to think flexibly about all the state-of-the-art technologies available and ideate ways to combine different technologies to create new value. I am looking for these two extreme types, but they are not easy to find. In that respect, I’d like to show people how hard Sony is working on AI—I think it will make them want to work at Sony.
Nagasaka：Robotics is complex technology. It is a matter of combining hardware, software, and recognition elements, so I think we need people who can demonstrate expertise in not just one area, but two or three. Also, it is important that people are aware of the issues and are highly motivated to break through these barriers. They need to be passionate about identifying issues and solving them on their own, and have the skills it takes to get that done—the ability to identify the issues and the ability to resolve them. We need people who have both of these skills. For those involved in R&D, there’s no point in just doing what your boss tells you. Individual researchers have to take the initiative to identify issues and resolve them by themselves, otherwise it’s very unlikely we would get good results.
Yokono：Kobayashi-san and Nagasaka-san have basically said everything I was going to say, but if I were to add something along the same lines of two desired traits, I would say that one, it is important that people in AI x Robotics have a global perspective. I think we want people who know about top-tier global developments. Second, I would say that we want people who just plain love to do R&D. I interview a lot of people, and I can tell in the first five minutes whether they really enjoy what they do. People who enjoy what they do take the initiative and dig into a subject, understand what the problems are, and use this in their own research. So I want to work with people who have a global perspective and simply love what they do.
Nagasaka：Doi-san, who created the original AIBO, often used the phrase “internal motivation,” talking about whether or not someone is driven by internal forces. I think it is particularly important that this internal motivation is sound, that is, directed toward society and customer value.
Kobayashi：The world faces a number of problems right now, and AI x Robotics is an area of technology that has the potential to resolve them all in one fell swoop. I would like to work with people who are able to find what it is that is rewarding to them.
