I hope to contribute to a
future where creators
can focus on exercising
their own creativity
Engineer Interview
Exploring the world of "sound"
with AI and signal processing
technology supporting
creators' challenges
Marco MartÍnez
Joined Sony in 2021
Job description: research and development
of AI and audio technologies
Fields of expertise: AI and Intelligent music
production
How did you get into your current
field of research?
Machine learning and signal
processing as
methods for
approaching the artistic world
I currently work on research and development of solutions based on artificial intelligence that can assist or automatically do sound mixing in film and music production. This is expected to reduce the workload of creators by easing the burden of mixing in the post-production process, which can be a complicated and time-consuming process. Although I started my research of AI and music production technologies during my doctoral program, as an undergrad student I majored in Engineering. The expertise I acquired through my studies in this field became the basis for my current research activities. As an electronic engineering student, I discovered audio signal processing and this led me to discover the world of music technology. I immediately knew that I wanted to work in this area and after obtaining a master’s degree in Digital Signal Processing and studies in Music Production, I learned to approach music from the technical and creative side. This made me aspire to contribute to the technologies that allow us to create, transform and deliver sounds.
Please share any formative experiences that have led to where you are today
Curiosity about "sound"
stimulated by daily life filled
with music
I have always felt a passion for music and engineering and, more specifically, a drive towards sound and its timeless characteristic of immediacy. I am fascinated by the power that sound has on our perception and consciousness, whether it is beautifully created music, natural sounds or artificial sounds such as those emitted by trains or machines. This started at a young age and was motivated by growing up in a family with a music-loving father, a vinyl collector, and a brother who was a singer for many years. Also, probably further fueled by my home country; Colombia, whose culture is always steeped in music and dance. So, in my teens, I learned to play the bass and for a few years I joined a Reggae band. Coming from a musically oriented family and being able to experience the world of music in the underground scenes of Bogotá, my passion for music grew while also sparking an interest in audio engineering at that time. I think I always had a feeling that I wanted to work with music and sound, and more specifically from a different perspective, since I knew I was good at mathematics and problem-solving tasks, I felt that I wanted to use my technical knowledge for music.
What made you choose Sony?
Hopes to be a pioneer of AI and
audio technology areas with Sony
Studying both Electronics and music production made me appreciate the immense complexity of all the tools we use to create, record, and deliver music, so I just wanted to be a part of the research associated with this field. More specifically, when I was studying music mixing, which I found quite difficult at first, it made me wonder if there were easier ways to accomplish this task, as it seemed too overwhelming at the time. This is how I discovered that Intelligent Music Production was an emerging field, and this was my main motivation for my doctorate; the investigation of tools that help or alleviate the tasks performed by music creators. When I finished my doctorate, I knew that the current research I was doing during those years was just the beginning and that there were too many exciting topics to explore and investigate. Sony's R&D Center then presented itself as the best option to pursue my research interests, since here I could continue my research activities and lead a relevant project. I feel that both Sony and I are pursuing research for creators and there are still many new things we can explore together. I am very pleased to have such opportunities.
What do you consider to be great
about the Sony R&D Center?
Involved from research to
application together with creators
I would say that one of my favorite things is how quickly the workflow is integrated between research and development. For me, it’s quite amazing to be researching a prototype and seeing its implementation and application by various entertainment studios and Sony’s products around the world. It is also unique and very helpful for studying the music or pictures related theme that we can have the opportunity to get feedback from various talented experts of Sony Group, such as musicians and producers of the music business and acoustic designers and mixing engineers of the pictures business. What is more, I’m currently I'm also involved in the projects involving film production, which is in collaboration with Stuttgart Laboratory of Sony’s R&D Center. That's very interesting because all my experience so far has been with music production or technologies, and it is new area for me.
What are your goals for the future ?
Broaden the possibilities for
creators and the next generation
I believe that technologies like AI for music have the potential to transform the lifestyle of music consumers and creators. One of our research results have showed that AI learn music post-production tasks and generate professional-level music mixes and I believe this is a major achievement in the field of intelligent music production. I hope to contribute to a future where creators can focus on exercising their own creativity by allowing AI to assist or automatically perform complicated tasks by evolving and merging conventional technologies by AI. As a project leader, I have also gained experience supporting the students who have interned with our team and it is really exciting to communicate and discuss directly with them. I like to be active in the scientific community, presenting papers at world-renowned conferences and participating in tutorials and workshops, so I would like to share my experiences and help many students to become researchers in the future.
What is your motto for
research and development?
“The intelligence is in the
sound”
“The sound of the rain needs
no translation”
(by Alan Watts)
The first one is a quote from research-related topic and is very representative of the research I do. I have been working directly with raw audio, or as we call as the audio waveform, and working this way, without any other representation or modality information, is a way of saying that everything we want to learn from the audio is already there; it's in the sound. The second one to say that there is something beyond words beyond the intellect, that can be conveyed with sounds. For example, when we listen to the sound of rain, whether you speak Japanese, English or Spanish, you don’t need to be told that it's the rain; you already know. That also means that in our perception, there is something that we understand directly from the sound, which is beyond words, beyond the use of the intellect, and I think these are quite profound words that resonate in many aspects of life.
