Collaboration
How the ORBEYE medical innovation came to be
Mar 29, 2019
Medical device notification number: 13B1X00277000634.
In surgical operations such as neurosurgery, spinal and orthopedic surgery, the magnification of microscopic tissue that is difficult to see with the naked eye enables a higher level of precision that directly contributes to safer and more effective surgery. The main role in providing such magnification is fulfilled by surgical microscopes.
Surgical microscopes have been mainly optical until now. However, optical microscopes require the surgeons to use the eyepiece for a long period of time (in some cases) and, as such, force the surgeons into an unnatural posture, which contributes to fatigue. On top of that, it is difficult for other members of the surgery team to understand what is going on at the surgical site in real time, which in turn makes it difficult to prepare for the next action and respond to spur-of-the-moment situations.
ORBEYE offers a new concept for such surgical microscope systems. Completely different from conventional systems, this system was made possible by the integration of Sony ‘s 4K and 3D imaging technology with Olympus Corporation’s expertise in the medical field.
We sat down with three Sony engineers that were dispatched to Sony Olympus Medical Solutions Inc. to work on the mechanical design, electrical design, and lens unit development, and asked them about the development history of ORBEYE.
Profile
-
Koji Fukaya
Imaging Product Development Department 2,
Sony Olympus Medical Solutions Inc. -
Masataka Kado
Imaging Product Development Department 2,
Sony Olympus Medical Solutions Inc. -
Yoshiyuki Kamata
Imaging Product Development Department 2,
Sony Olympus Medical Solutions Inc.
The realization comes afterward
──What kind of role did each of you play in the development of ORBEYE?
Yoshiyuki Kamata：I was mainly in charge of mechanical design.
Masataka Kado：I was involved in the development of the lenses, which entailed the optics and the mechanics and controls of the lenses.
Koji Fukaya：I was in charge of the electrical design and systems.
──Would you say you had a different mindset in designing a medical product than you did when you were designing consumer products at Sony?
Fukaya：In the sense of designing electric circuits, what I do is pretty much the same regardless of the product. However, it was quite different in terms of the standards we had to comply with and in terms of risk management. for instance, considering how the product needed to behave in the event of a failure. At first, I was under the impression that I was simply going to have to do more work, but in reality, as I observed the actual medical sites using ORBEYE, I realized just how critical risk management was going to be, which led to a change in my mindset.
Kado：I also gradually changed my mindset. In the beginning, there were many days of contemplating what kind of product to make from basically scratch. I was very focused on development. Then, when I attended the installation of ORBEYE at an actual clinical site and saw it being used in actual surgery, it really struck me just how important my work was. There was nothing quite like seeing it being used to save the lives of real people.
──During development, did you take time to visit hospitals to talk with actual surgeons?
Kado：The ORBEYE project had already been incubated by the Olympus team, and there was already an HD (High Definition) prototype when we joined the development team. So, when I started with Sony Olympus Medical Solutions, I did not need to do such footwork such as visiting surgeons for research. Rather, I needed to focus on refining the prototype.
Since it was a joint venture, our focus was on what kind of value we could add by joining the team. Take the camera, for example, we needed to think hard about how to upgrade the prototype originally conceived of by Olympus. That’s why we went through so many desktop evaluations. For example, we would have surgeons come to a meeting room and check out our development models so we could get their feedback. We tried to capture every detail of their words to understand and figure out what the real voice of the customer (VOC) was, and then translate that into our design. We iterated this process as many times as we could to refine our specifications.
Fukaya：The difference, for me, between developing a medical device and developing a consumer product, was that I was no longer an end user of my own product. In other words, I was not able to confirm for myself that the functionality, performance, and specifications were sufficient from the end user’s point of view. So, it was critical for me to get feedback from actual end users—surgeons that is—in order to judge whether the product I was working on was truly valid or not. So, I made it point to repeatedly get feedback from surgeons to polish the product.
It made me very happy when the surgeons tried out what we had completed and finally said, “Now this I can really rely on in the operating room!”
Kamata：Of all the feedback we received from the surgeons, the one time that stands out the most was when we first showed them a 4K 3D image. You see, it was still at an early stage of development, so the machine hadn’t taken form yet. We just had all the devices and components laid out on the table, but the surgeons were very impressed with the image! I felt confident at that point that we would be able to satisfy the VOC.
We had heard that the surgeons had indicated that the product under study at Olympus prior to the formation of Sony Olympus Medical Solutions still had a long way to go in terms of image quality. So, this moment brought home to me for the first time the significance of our involvement in this project and just how great a value our product would deliver.
Where did Sony technologies contribute?
──What Sony technology was instrumental in bringing ORBEYE into life? Specifically, what kinds of technologies were incorporated into the prototype that Olympus had?
Fukaya：For this product, we applied Sony digital imaging technology in every process—from implementing the lighting, to capturing images with the imager, to transferring and processing those images and displaying them on the monitor.
For example, take how the 4K image feed is processed. Since it is a medical device, the key point is how quickly it can process that huge 4K 3D data and display it on the monitor. That is, if there is any time lag between what the surgeon is doing and seeing on the monitor, obviously that could impact the operation. So, it is critical to find a way to keep such latency, if any, as short as possible.
In terms of image processing, the more you process, the better the image quality. However, the more you process, the more time it takes. As a means to address this issue, we implement so-called “hardware/software co-design.” This was about balancing how much image processing to do to obtain high image quality and how much image quality to secure with the optical components, to make the system, all in all, optimal for actual surgery.
Kado：One of biggest challenges we faced was that of making the ORBEYE compact. Traditional surgical microscopes, the optical ones, tend to be very large. With the optical design being handled by engineers from Sony in this Sony Olympus Medical Solutions joint venture, we were able to leverage Sony’s know-how from designing compact consumer products, and we succeeded in reducing the size needed for the optical system in ORBEYE.
Since ORBEYE has a counter-balance arm, the size of the microscope is determined by the size of the optical system, which in turn determines the size of the arm and the body of the product. In other words, it was an absolute must to downsize the optical system in order to make the overall product compact. That is one aspect in which Sony technology helped make this product possible.
Struggles unique to medical equipment and unique to a joint venture
──What were some of the biggest challenges you faced on this project?
Kamata：This project took about 5 years, so there were a variety of difficulties. In fact, it was the first time I experienced so many struggles all in the same project. They were a different flavor of struggle compared to my previous department, where I was usually under pressure to put out consumer products in a short period of time to meet the short life cycles of such products.
First of all, from a mechanical design point of view, I struggled with the weight of this product. That is, up until now, I typically have dealt with products that weighed up to 2 kilograms, and some as light as 100 grams. The ORBEYE weighs about 200 kilograms, with the heavier components weighing about 20 kilograms themselves, meaning that they could bend under the stress of their own weight. With small products like I was used to dealing with, I never needed to worry about deformation due to the weight of the components. So, for that reason, I needed to completely change my mindset as a mechanical designer. It was very difficult to deal with the large size and trying to improve the operability in light of that.
Also, there was the struggle related to this being a joint venture. For medical devices, the factory is completely responsible for shipping. So, for this product, there were all manner of different parties, not just us in R&D, but also those from Sony and Olympus manufacturing and those from Olympus sales & marketing divisions. In other words, just with pre-shipment evaluations, we needed to collaborate in a completely different culture than we were accustomed to. It was a pretty big challenge sometimes to ensure that we properly understood each other.
Fukaya：That’s true. There was a difference in cultures between Sony and Olympus, and on top of that, Sony was taking on the challenge of developing a product in a medical field that it didn’t have any experience with. So, we needed a great deal of guidance from Olympus and the team members seconded from Olympus throughout the project. It was a real challenge to be sure to exchange information closely as we proceeded with development.
Kado：I was involved with the development of the lenses, and in terms of exchanging information, ironically, I had a hard time working with team members at Sony.
When designing something, you have to show and share with other designers the background, the goal and purpose, etc. However, since this was a medical device and something that Sony had never undertaken, it was hard for me to convey to team members back at Sony the kind of product we were trying to create, who would be using it, and how it would be used. So, I struggled quite a bit early on trying to get us all on the same page about the specifications the product required. I personally was not very familiar with the medical field, so, I was learning myself, then needing to educate others about what I had just learned, right away. So, this was another challenge.
Thoughts on next steps
──ORBEYE has just been successfully released to the market. Now what? What are you looking at next?
Fukaya：I think there are still many things we can do. Of course, we will continue to improve ORBEYE. Also, we can explore various different approaches. Because this is an imaging device, the key is to find a way to obtain a large quantity of image information from the surgery sites and process it to provide it as beneficial information back to the surgeons. In that regard, I believe that there is so much we can do to contribute with Sony’s imaging and sensing technologies.
Also, I think that there are many things we can do in terms of usability to make this product even easier to use, which in turn will make it better for the patients, so we need to be vigilant in that regard as well.
Kamata：With the development of our first model, we were able to propose an approach different from the conventional microscope. With this new product we have provided new value that will act as a game changer in the operating room. For example, one of the types of new value it provides is allowing multiple people to view what is going on at the same time, achieved by replacing an eyepiece with a display. This is unlike the conventional manner of just one person at a time peeking through an eyepiece. We have already received favorable feedback. However, there are many things that we still weren’t able to do with this first model, and there is a plethora of technologies in the Sony toolbox that we haven’t leveraged yet. So, I definitely want to revisit the question as to whether the way it currently is is the truly the best as we apply other technologies and approaches.
In this development process, we pursued an “optimal” form for the product based on the prototype we had when we started. However, the definition of what is optimal will surely change, depending on the technology we incorporate. The development span for products in the medical field is pretty long, so we need to look 5 or 10 years out to anticipate what technologies will be out there, and then develop the next generation of products based on this outlook.
Kado：When we were working on establishing the specifications for ORBEYE, there were various functions that we had to give up for reasons such as lack of development resources at the time. As Kamata-san mentioned, there are various technologies in the Sony toolbox that we have yet to tap into, and even in the fields of optics, sensors and signal processing, there are so many undiscovered technologies. So, I am currently working very intensely to figure out what medical needs we can address by leveraging these untapped technologies and to identify areas for improvement.
In order to discover those needs, I am gathering a variety of feedback from the surgeons involved in the clinical evaluation. The surgeons are pretty good at providing accurate comments, but we will make every effort to decode the true needs hidden behind their comments, and then leverage Sony technology to address the needs we discover in future development. That is the value Sony Olympus Medical Solutions has to provide.
──What kind of feedback have you received so far about ORBEYE?
Kado：We have been told that ORBEYE is also beneficial in terms of education, as it makes it possible for everyone in the operating room to see the affected part on the monitor.
Fukaya：Also, it was well received by the nurses and other staff. Until now with the eyepiece lens type devices, only the main and assistant surgeons were able to directly view the image. The rest of the staff would have to view via a video monitor, but the image quality left a lot to be desired. With ORBEYE, everyone can see exactly the same thing as the surgeons, so it is easy for everyone to follow what the surgeons are doing and anticipate the next moves they need to make. This makes it much easier for them to support the operation.
Kamata：Another favorable comment was that the mobility range of the arm is much better than that of conventional devices. When I observed the surgeons in action move the arm at will, I realized that they interact with the arm in many different ways to examine the affected area depending on the type of operation they are performing. I had thought I understood the needs of the surgeons, but I realized that the surgeons do more with the arm than I had originally realized.
The significance of Sony’s entry into the medical field
──I don’t know if the word user experience (UX) is appropriate for products in the medical field, but do you think it will be possible to change how surgery is performed by incorporating technologies such as augmented reality (AR) and artificial intelligence (AI)?
Kado：That’s my dream. ORBEYE allowed Sony to get to the center of the operating room. So now, I hope that we can leverage that positioning to link a variety of devices together, thereby transforming the operating room. In order to make this kind of change happen, we don’t want to simply continue to make the next generation as standalone units. Rather, we need to explore various possibilities.
Personally, whether AR or AI or some other technology, we definitely want to leap ahead about three steps past simply making the next ORBEYE. We want to come up with products that the industry will take note of as being possible only because of Sony Olympus Medical Solutions’ manufacturing creativity and unique approach that no other companies have.
──In order to make such new proposals in the medical field, what kind of skills and mindset would you look for in new members?
Kamata：I would look for new team members to be the type that are curious and quickly intrigued by a variety of things as they pursue their work. That is not to say we don’t already have people like that [laughs]. The reason that characteristic is important for the work we do is because Sony has so many technologies that can potentially be applied. Also, there are very many needs in the medical field, although they may be hard to see at first glance. It takes someone with the curiosity to explore and uncover those technologies and needs, or they would probably not notice them. So, that is why for this industry, I feel we need people that can maintain intrigue and curiosity in what they are doing.
Fukaya：I have something similar to say on the matter. At Sony Olympus Medical Solutions, you can do a lot of things. Just because the job title is engineer, it doesn’t mean that is all you do. You need to get to the actual site, research it, interview end users, talk technically with the researchers, and so on. There are many aspects to being a good engineer. So, the kind of person that can take the initiative in working within a wide, diverse field is the kind of person best suited for our work.
As a company, we have diversity in the sense that our members come from a variety of divisions. So, I think people who can enjoy that diversity will enjoy the work we do.
Kado：By all means, I would like many types of people to work with us. That is because development at Sony Olympus Medical Solutions requires diverse perspectives. I agree with what Fukaya-san and Kamata-san said, in that there aren’t many places like this where you have the freedom and flexibility to say “I want to do this” or “I want to do that” and actually get to do it. Sony Olympus Medical Solutions has a great culture for accommodating your needs and wants in the pursuit of your career goals.
Sony Olympus Medical Solutions Inc.:
Established with the goal of forming a 3D/4K surgical imaging business leveraging Sony Corporation’s imaging technology and Olympus Corporation’s optical technology and medical business expertise. Started developing surgical medical devices in 2013. Developed a 4K surgical endoscopy system in 2015, and the ORBEYE 4K 3D surgical microscope system in 2017, both of which were released by Olympus.
Sony Group’s engagement in the medical field：
Leveraging imaging and other technologies from the electronics field, Sony has been developing various types of medical imaging equipment since the 1980s, including printers, cameras, recorders and monitors. Sony also provides cell analyzers/sorters suitable for researchers in advanced research fields such as immunology, oncology, and regenerative medicine.
In the field of medical equipment manufacturing, based on healthcare professionals’ voiced desire to perform high-precision surgical procedures utilizing high-definition video, Sony responded by proposing a full suite of high-definition medical imaging equipment that offers recording, transmission, and playback at 4K resolution to support surgical performance in the operating room.
In an alliance with Olympus Corporation and Sony Olympus Medical Solutions Inc., the Sony Group’s engagement in the medical field has led to various achievements such as the development of the ORBEYE 4K 3D surgical microscope system.*
* Manufactured by Olympus Medical Systems Corporation (with the exception of the monitor and recorder); monitor and recorder manufactured by Sony Corporation
