People
Series: Inside the Minds of Sony’s Corporate Distinguished Engineers #1 Yoshiyuki Kobayashi
Making Machine Learning technology available to everyone
Aug 20, 2021
In this series, Sony’s top engineers discuss their careers, research, and their ideal profile as an engineer at Sony.
In #1, we hear from Yoshiyuki Kobayashi, who is working to promote AI development environments.
Corporate Distinguished Engineer
Sony certifies its engineers as "Corporate Distinguished Engineer" who formulate and execute technology strategies while identifying signals of change, and support the development of talent in order to ensure Sony’s sustainable growth.
Profile
-
Yoshiyuki Kobayashi
Principal Researcher
Distinguished Engineer
Tokyo Laboratory 19
R&D Center
Sony Group Corporation
Began programming software in junior high school and also passionate about music.
Yoshiyuki Kobayashi was born in Mie Prefecture, central part of Japan. He grew up in a relaxed rural environment. He loved math and science, which he tackled like puzzles to be solved.
As a young child, I remember seeing heavy machinery clearing footpaths between rice paddles and thinking it was incredibly cool. Looking back, that sparked my interest in machines. I remember writing “I want to build giant robots in the future” in a kindergarten essay.
In elementary school, I became obsessed with video games, and as I played, I began wanting to create my own games. When I entered junior high school, I got a Sony’s MSX computer and started programming. At the time, there was a magazine where people could submit their own programs, and my posts have been published many times. In high school, I sold some of my software, including games and utility tools, through the software vending machines that were available at the time, and after graduation, I received a letter from a video game company trying to scout me (laughs). At that point, I already knew that I wanted to pursue a career in the sciences. On the other hand, I was also passionate about music. I learned to play the piano when I was a child, and I continued my musical activities through high school and university, so that was another important part of my life.
As a student, Kobayashi was obsessed with programming and electronic music, while he wasn’t very interested in his classes
From around junior high school, I didn’t really pay attention in class. I spent most of my lessons creating pixel art for characters or composing music. I wanted to pursue a career in the sciences, but I don’t think I really understood what the sciences entailed (laughs). To be honest, it was only after I joined Sony that I really understood differential and integral calculus, and for a while, I often had to pull out my old textbooks to relearn them. On the other hand, I had a fair amount of self-taught knowledge about audio waveform processing, which is essential for electronic music composition, but I didn’t even know what the field was called until after I joined Sony.
In university, I immersed myself in composing electronic music. Eventually, I began to question the complicated process of inputting sounds and the need for special, expensive equipment, and I began to think about creating a system that would allow anyone to compose music with ease. So, I released my own DAW (Digital Audio Workstation—a system used for recording, editing, and producing audio files) software for free, and it was downloaded hundreds of thousands of times. I was thrilled at the idea of people using my tools to make music. It’s a bit of an exaggeration, but I thought that I was contributing to the arts. I wanted to engage with people who were motivated enough to tear down the barriers in front of them. That is a concept that I still cherish to this day.
Music has been an important part of his life since childhood
When a Sony employee found out that Kobayashi was creating his own DAW, he invited him to work at Sony.
When I was a student, I happened to mention to a Sony employee that I was making music software, and he invited me to work part-time to develop a professional sampling reverb called DRE-S777. Later, he also invited me to take the company entrance exam. Since I wasn’t too serious about my studies, I didn’t think I actually stood a chance of working full-time at a company like Sony. However, the people at the office were quite interesting. There were excellent generalists who could handle anything, and there were also many eccentric characters. And yet, the overall average score was extremely high. I was attracted by the diversity of the human resources, and the open-mindedness of the people at Sony, which still hasn’t changed today.
From elemental technology to further upstream. His first encounter with Machine Learning.
Kobayashi joined Sony as a full-time employee in 1999. He worked on signal processing and program development for professional audio. Later, due to an organizational change, he was transferred to a Research Institute.
After transferring to the Research Institute, my focus shifted from developing music production tools to researching audio signal processing. I worked on the development of echo canceller technology, which is used in online conferences, and 12 tone analysis technology, which analyzes music waveforms, and it made me realize how interesting the fundamental upstream technology is.
Sony’s proprietary 12 tone analysis technology analyzes the twelve notes of the chromatic scale to recognize the “mood” of a song and other qualities. By automatically extracting and categorizing the metadata of a large quantity of music, it can instantly generate a playlist of songs that the user will like.
It was a very satisfying job because I was allowed to work freely on my own projects. I gradually attracted more and more collaborators, and eventually, the technology was adopted for various projects within Sony group.
When he transferred to a Research Institute
The 12 tone analysis technology is an excellent example of recognition technology. Kobayashi became fascinated by the possibilities of Machine Learning.
To recognize musical attributes such as chord progression, genre and tone, we had to use Machine Learning, a technology that is currently attracting a lot of attention. When I first began working with this technology, I remember being impressed by its huge potential. If we could apply it to other fields, what wonderous things could we accomplish? I foresaw a future where various intellectual tasks would be handled by AI.
Machine Learning and neural networks themselves have been researched since the 1960s, and Sony has been working on them from a relatively early stage. Sony was the pioneer in the industry to introduce voice recognition functions for car navigation systems, and it was also used in robots such as the “AIBO”. When I developed the 12 tone analysis technology, I received a lot of advice from people working on those projects. I didn’t even know the term “Machine Learning” back then, so it was very helpful.
Basic technology is like math. We have to search for the answers together.
Kobayashi’s long challenge of making Machine Learning available to everyone had finally begun.
It was clear that companies that tackled AI research early and put it to practical use would have a competitive edge in the next generation. I started researching and developing Machine Learning technology in earnest around 2006. Since then, we have been working to make it easier to use, leading to the creation of tools such as the Neural Network Console.
What was difficult at the time was that there was no substantial consensus within Sony on what kind of future would be realized by Machine Learning. Of course, there were many people who supported me, but there were also others who were skeptical about my technical background and asked, “What can we possibly do with Machine Learning?” But I was thick-skinned (laughs) and just carried on emphasizing the importance of Machine Learning. In particular, I focused on education and support activities, such as inviting lecturers to give technical training on Machine Learning and accepting personnel from other departments to provide on-the-job training and work as evangelists. As a result of our efforts to support not only Sony’s main product areas such as TVs and audio equipment, but also other areas where there is demand, our tools are now widely used at factories and other manufacturing sites.
I think one of the reasons we have been able to continue promoting Machine Learning is that Sony has a culture of “Bottom up activities,” where employees can voluntarily engage in research and development that interests them.
Image of Neural Network Console
“My contribution to Sony has been my continued research and development in Machine Learning.” He explains the direction of his R&D as follows:
I think that the closer we get to basic technology, the less we can rely on a single company’s technologies and expertise. Instead, like in the world of mathematics, we should all work together to discover new formulas. I believe that Machine Learning will continue evolving through open innovation, with the whole world helping to realize more useful technology.
So, instead of proceeding with the development alone, I think Sony should share information with the outside world. As long as we do this, people who excel in this field will naturally be drawn to us. It is important for Sony to become a place for people who share these goals.
In addition, one of the strengths of Sony’s globally diversified business operations is that we have professionals in many different fields. By collaborating and combining technologies, it opens up a wider range of possibilities. That is a major advantage. In particular, I’m excited to see what Sony can do in the entertainment business in a future where Machine Learning is widely used for intellectual exploration and the arts.
Machine Learning will make the world a happier place
Will Sony be an attractive workplace for those involved in Machine Learning and other basic research? What are your expectations for the new people you will be working with?
Machine Learning is still in its infancy, and we are only starting to discover its applications. From our point of view, Machine Learning is just computers and software. The question “What can we do with Machine Learning?” is just the same as people asking, “What can we do with computers?” a few decades ago. The answer is “anything.” It is safe to assume that any simple intellectual task that is repeated many times can be replaced with machine learning. How to expand the range of applications will be a major theme for me moving forward, and I am convinced that there is a lot of demand in the Sony group.
If you are someone who understands that Machine Learning will dramatically change the world, then I would love for you to come and work at Sony and help us get a head start in this transformation.
In a way, this is just the latest in a line of technologies that started with semiconductors and has continued on to electronic circuits, computers, software, the internet, and AI. Perhaps the next big thing is something that I am completely unfamiliar with. I’d love to work with someone who can passionately tell me, “Deep learning is old hat. The next big thing is this!”
We are no longer in an era where you can complete your life’s work at only one company, and your personal interests will change as you gain experience. I believe that our mission is to create a workplace where people can channel 100% of their abilities into their interests. There is no specific goal, but we will continue to improve, doing our best to become a company where people can accomplish their dreams.
Lastly, we asked Kobayashi about his dreams as a Corporate Distinguished Engineer. The keywords are “shortening” and “acceleration.”
My goal for the future is to make Machine Learning and AI, which are currently in their infancy, available to everyone, not just a select few developers. Some say that AI will steal jobs or hurt people, but in my opinion, this is nonsense. I am convinced that if we can shorten the time to a future where the use of Machine Learning is commonplace by even a year or two, the world will be a happier place for it. This is our immediate mission, and we will continue to work on developing tools that allow anyone to utilize AI.
Long-term, the question is how quickly we can identify new technologies that lie beyond Machine Learning and implement them in society. More advanced technology will surely make the world a happier place. My life work is to shorten the period of social implementation and accelerate the evolution of humanity. I will continue striving to reach this goal.
Kobayashi’s remote working environment at home.(Right image)
