Collaboration
Will "Flying Robot" Change Society?
- The Next Generation Drone Company Opens
the Door to the Future
Oct 31, 2019
The drone company Aerosense is a joint venture between Sony and ZMP. This company, which develops everything from hardware, such as multicopters and VTOL,* to cloud services, is working on drones that can fly in a narrower space, for a longer time, and over a longer distance, aiming to solve social issues. I asked members of the company about their present position in their activities and their future prospects.
*VTOL: Vertical Take-Off and Landing Aircraft, which is a fixed-wing aircraft that can hover, take off, and land vertically and autonomously.
Profile
-
Kohtaro Sabe
President & CEO,
Aerosense Inc. -
Kosuke Suzuki
Aerosense Inc.
-
Jun Yahata
Aerosense Inc.
One-stop development from hardware to analysis of drones
──Could you tell me the story behind why Aerosense was established?
Kohtaro Sabe：From the hardware, that is, the drone itself, to autonomous control and positioning technology and cloud services for data analysis, we are consistently developing the functions required to utilize drones. Specifically, we are dealing with four types of products: multicopters called AEROBO; VTOL-type drones; a surveying instrument used for ground control point marking and control point survey, called AEROBO Marker; and cloud-based data processing service, AEROB Cloudthat automatically generates 3D data by simply uploading images.
Kosuke Suzuki：When you upload an image captured by AEROBO to the AEROBO Cloud, you can generate 3D data in about an hour, which would usually take half a day to a whole day on a desktop PC. I think it's practically a big difference from other companies in terms of ease of use, time, and precision all achieved.
──By the way, what is the control point survey?
Jun Yahata：When you create maps and drawings, it is necessary to use control points that are embedded in the worksite ground as the starting point of all types of survey. And a control point survey is a survey to determine and set those control points. The AEROBO Marker has a GPS in it, and by performing positioning and correction process between its GPS system and electronic control points, which are GPS-based control stations managed by the Japanese government, it enables measurement with an error of 1 cm or less. High-precision three-dimensional data is created by combining this positioning data with aerial image data. Measurements have been so far carried out using extremely complicated surveying instruments, but the AEROBO Marker has been well received for being able to allow the user to conduct high-precision surveys simply by placing a marker and pressing a button.
I wanted to build a flying robot.
──Could you tell me the story behind why Aerosense was established?
Sabe：We were originally developing a "flying robot" at Sony Corporation (hereafter “Sony”), but we did not have a platform or infrastructure to commercialize it. Under such situations, we paid attention to ZMP Inc. (hereafter “ZMP”), which provides automated driving technology as business-to-business. To combine their experience about the robotic platform and our technology of flying robots, we decided to join forces.
──When did Sony begin researching a "flying robot" and how has this research progressed up until now?
Sabe：In 2012, I started research as a project at the Business Design and Innovation Laboratory (BIL). After working on the first AIBO, I was away from robotics for a while, but I began to think I’d like to try practical robots next time, which made me aim at flying robots, that is, drones.
In 2013, the budget for making a prototype was finally approved, and we conducted indoor experiments with small prototypes. The idea of joint development with ZMP was coming up in 2014, and 5 people from Sony and 5 people from ZMP gathered for working out the concept, visiting construction companies to conduct interviews, and performing demonstration experiments to confirm the feasibility of our business. Then, we established the company in 2015.
It took about a year to build a commercial version of the drone and cloud system, which is now mostly used in surveying.
From civil engineering to architectural engineering
—beyond the wall of precision with an error of 1 cm
──Tell us about the situations in the survey market and what initiatives Aerosense is taking.
Suzuki：The domestic construction and civil engineering industry is thriving. There are a lot of jobs, which accordingly increases surveying jobs. The labor force, on the other hand, is decreasing steadily. The survey market is not going to be a huge one, but it's going to be a growing market where more drones will be used to complement fewer workers.
Yahata：In the past, the surveying industry was facing aging workforce, but new technologies like drones have made it easier to hire young people.
Sabe：In the case of civil engineering work, survey is required at various stages. There are land survey before construction, control point survey in the drawing phase, volume survey during construction, and finished form survey after completion. When I started this project, I never thought that I would become so knowledgeable about surveying, but I think this is the result of my relationships with people in the industry, trying to closely communicate and propose various ideas to support them using our technology.
Suzuki：In the field of civil engineering, drone surveying has advanced considerably, and we have come to the point where we can compare drone images with drawings. In the future, the technology trend is to enter the field of architectural engineering by combining drone surveys with a tool called Building Information Modeling (BIM), which creates a 3D building model. I think the challenge is how to catch up with it and create markets.
Sabe：In order to get into the realm of architectural engineering, there are some issues to be solved. Firstly, civil engineering works can go with a precision of about 1cm, but a building that tilts 1cm cannot be accepted, and therefore, more strict accuracy is required. Secondly, drones can easily fly in large construction sites and its safety management is easy. But if you want to fly them between buildings under construction, drone technology must be further developed to enhance flight safety.
Yahata：To do this, we need the technology to approach closely to objects and avoid collision, while flying, even without GPS. I think the most possible candidate is the flight technology that uses a camera for monitoring surroundings and confirming its own position. Small size is also necessary.
──What measures do you take to prevent accidents and human errors?
Sabe：For the autonomous type, there is not much work on the operator side, so users tend to be less attentive. Therefore, we use special care in designing the self-diagnosis process to be activated when the drone is turned on and to check everything is working properly. Drones have various kinds of sensors, and it would take a lot of time for humans to check them, one by one. Our drones are designed to make a self-diagnosis, such as whether the GPS, axis sensor, accelerometer, and barometer are all functioning well, while being carried around by the user. Such a hidden mechanism is automated in order to improve safety.
Many things related to drones are invisible. We can’t see wind, air, atmospheric pressure, radio waves used in telecommunications and GPS, and so on. In addition, drones use the geomagnetic field to recognize the direction, so if the geomagnetic field is distorted for some reason, it will be very troublesome. It's important for all sensors of a drone to work properly to sense and determine those things.
Possibility of "real-time transmission of uncompressed
4K video" with wired drones
──I heard that there is a wired type of autonomous AEROBO drones. How was it developed and what kind of use is it supposed to provide?
Sabe：“Longer hours flying,” “greater distance flying,” and “indoor or narrow space flying” are three things that cannot be easily achieved by existing drones. We have been finding a way out for these challenges, and the wired power supply was our answer to address one of them: “longer hours flying.” By taking advantage of the benefits of wired power supply, we are anticipating a new application in which uncompressed high-quality video is delivered live for long periods of time.
Suzuki：This comes from a small drone that we developed when we were at Sony. Its battery ran out quickly, so we decided to make it a power-supply system and started working on a prototype. I also applied for intellectual property registration in the hope that it could someday develop into a business opportunity.
After we moved to Aerosense, we discussed and considered, "If we put a generator on a ZMP self-driving car and connect it to a drone, the drone can fly anywhere." From there, we began to build a power supply that could fly large drones. But simply sending electricity isn't fun. So, to make it a Sony-only product, we decided to put an optical fiber in the cable so that it can transmit the best content captured by a drone. To leverage that feature, we thought we need to provide a zooming function, a gimbal, and a high-quality camera... In this way, the wired drone has gradually evolved to be the current one.
Yahata：The optical fiber enables the transmission of uncompressed 4K video in real time. At the Japan PGA Championship (a major golf tournament in Japan) held in July this year in Ibusuki, Kagoshima Prefecture, video shot with this wired-type AEROBO drone was aired as a live television program. On the last day of the tournament, the drone stayed in the air for about eight hours at an altitude of about 60 m, and continued to send 4K images.
Sabe：Basically, drones should not affect players’ conditions, so they must either fly very high or not fly at all in the first place. If a drone flies up and down during the game, its noisy sound or sudden appearance coming into view will distract players’ concentration, which might cause an unfair game environment between players. So, once a drone leaves the ground, it is required to keep on flying for a long time.
──It's important to advance into new fields and build up results one by one. What other uses can this wired type have?
Yahata：Because it can see from a high fixed point for a long time, it can be used for security or surveillance purposes to see a wide area outdoors. For example, I think it can be used to monitor ships arriving and leaving at the port or to guard the construction site.
Sabe：I think it can also be used in the event of a disaster. Local governments or any other organization can use the drone to quickly check the situation by looking down on the city from a high point. In fact, we are conducting demonstration experiments, on the assumption that Mt. Aso has erupted, by shooting 4K video through the zooming function and distributing it to various locations using 5G.
The future of drones realized by VTOL through deregulation
──What about VTOL? How do you see it in your business?
Sabe：When we started the joint project with ZMP, we decided to do something new because we can't differentiate our products with ordinary drones alone. So we made VTOL drones as a joint research with Kobe University, which was doing research on VTOL at the time. We received a lot of advice from them.
Because the VTOL has a fixed wing, it generates lift when it moves forward. Unlike a multicopter-type drone that consumes a lot of energy just for floating, the VTOL can keep floating with the energy by moving forward so that it is much energy efficient in terms of flying over a longer distance.
However, we still have regulations on drone operations, requiring to fly only within the operator’s visual line of sight. These regulations are going to be gradually relaxed, though. Starting from 2018, drones are allowed to fly beyond the visual line of sight in unmanned areas, and by 2022, they can fly beyond visual line of sight even over manned areas. I think it's time for VTOL to play a big role.
──What kinds of needs do you expect for VTOL?
Sabe：From the beginning of this project, we were considering the emergency delivery of goods, such as pharmaceuticals, and we believe that this is actually feasible. There is also a need for delivery business such as regular service to remote islands and underpopulated areas. In addition, I think there is a possibility of use in long distance inspection. Flat infrastructures, such as construction sites, solar panels, and roofs, can be measured flatly in a zigzag pattern, which is what current multicopters are good at. However, for inspecting linear infrastructures, such as roads, railways, power lines, and rivers that are usually tens of kilometers in length, multicopters are not suitable, and I think that VTOL will play an active part in such areas.
Currently, if a 20 kilometer power line or river needs to be inspected, 10 teams, each consisting of three engineers, are dispatched in 10 vehicles, check each assigned section, and then come back to report. If this kind of tasks is taken over by an unmanned drone, it should bring enormous cost-effectiveness.
What is a drone developed based on Sony's wisdom?
──What are the characteristics of "Sony's unique drones"?
Sabe：First, Sony’s drones have a Sony image sensor. In addition to image sensors, Sony Imaging Products & Solutions Inc. is developing multi-spectrum cameras as modules and has just begun applying them to agriculture in the United States. Aerosense is planning to collaborate on this effort by providing the cloud system and analytics support.
By being able to perform multi-wavelength analysis, our drones can help check vegetation, the distribution of fertilizers and water, and the degree of variability in crop yields, and I think this is an area where needs will continue to grow.
Suzuki：We have been working with the ELTRES* development team in Sony Semiconductor Solutions Corporation since we were at BIL, and we are conducting demonstration experiments with them together. Japan is now paying growing attention to the necessity of keeping track of where drones are flying, and together with the ELTRES technology, our drones can be easily located wherever they are.
* ELTRES: Sony’s unique wireless communication standard that has the features of “robust long-distance communication,” “high-speed mobility communication,” and “lower power consumption.”
Yahata：Also, since the base part of AEROBO is manufactured in Sony's factory, Sony’s know-how on mass-production design and quality-reliable manufacturing, including dust-proofing and waterproofing, is utilized.
Suzuki：It would be interesting if we could create new functions by utilizing technologies developed in business units and R&D organizations of Sony, such as semiconductor, camera, and communication network technologies.
I also want to make our drones smaller and lighter, so I would like to continue working with Sony, which is good at it.
──Lastly, please tell me the significance, fun, and future prospects of working at Aerosense.
Suzuki：In surveying, monitoring and inspection, agriculture, and logistics, there is a shortage of human resources, and if such a situation continues, this labor shortage will become increasingly severe. I think drones can be a major solution to this problem. Also, when a disaster occurs, there are many places where people can’t go or do things without a drone. In that sense, the development of drones is significant in contributing to society.
The Ministry of Land, Infrastructure, Transport and Tourism is driving the committee to talk about the use of delivery drones as a new logistics business model, and the public sector and private sector are discussing together whether it is possible to deliver mails to post offices in underpopulated areas on mountains. Aerosense is participating as a member of this committee and offers various opinions to the government, which is very interesting part of working in Aerosense, as well.
And on top of that, I find it very interesting that we can make products and services by listening to each customer's voice and then deliver them directly to our customers.
Sabe：We have built up various know-how by providing total solutions in the field of surveying. Using this experience and knowledge, we’d like to continue to explore new fields.
