Cutting Edge
Aiming for More Comfortable and Secure Mobile Society with AI and Sensing Technology
Future Mobility Project
Aug 31, 2020
Under the theme of "Future Mobility Project," Sony is collaborating with Minnano Taxi (Everybody's Taxi) Corporation to explore it's way of contribution in the mobility field using AI and sensing technology. As part of this project, we launched a demand prediction service for taxis in November 2019. We are also aiming to create further services that lead to the development of safe driving support tools, and a more comfortable, more safe and secure mobile society. We spoke to Hiroki Takakura, who has been working on safe driving support tool initiatives, and Takahito Migita, who is engaged in the development of demand prediction services, both from Sony Corporation's AI Robotics Business Group.
Profile
-
Hiroki Takakura
Cloud Service Development Dept.
AI Robotics Business Group
Sony Corporation -
Takahito Migita
Cloud Service Development Dept.
AI Robotics Business Group
Sony Corporation
1. Exploring technologies and data
that can contribute to safe and secure driving
──What sort of initiatives have you been involved in regarding support for safe driving?
Hiroki Takakura：We have been focusing on the search for technologies and sensor that data will actually contribute to safe and secure driving. More specifically, we have been using a test vehicle equipped with nine image sensors, as well as solid state LiDAR and IMU acceleration and angular acceleration sensors. The test vehicle is designed to accurately recognize a 3D space, putting together CAN data related to the driver's operations such as acceleration, braking, and steering wheel, with sensor data concerning the environment surrounding the vehicle, and then analyzing it.
Collecting data concerning the environment surrounding the vehicle in the form of pictures, along with data related to the driver's operations
──What stage are you at right now?
Takakura：Currently, we are collecting data through test drives. To acquire this data, we asked a professional driver who has chauffeured Sony's executives for ten years to drive the test vehicle. I have also been in the car with him, and as a result of his acceleration, deceleration, parking abilities, and ability to react to unexpected situations, it was a very different experience. Discussion on subjects such as, what you need to pay attention to when rapidly assessing your surroundings when driving, and his opinions on what makes a good driver based on his many years of experience as a professional were extremely helpful.
While these data are qualitative, I believe that quantifying comfortable driving from a qualitative sense and reproducing it using technology is an important theme.
──How will you go about analyzing the data gained from the test vehicles, and using it to support safe driving?
Takakura：For example, if we can quantify the "comfortable driving" that I was just talking about, we can provide feedback to drivers. Taking this even further, control assistance that can help drivers achieve this level of comfort may also be possible.
Currently, our experiments are based on the test vehicles, but we believe that acquiring a lot of sensor and driver operation data through our partnership with Minnano Taxi will help us accelerate various kinds of technology development.
An image showing what LiDAR captures when installed on a test vehicle
──What can you tell us about future developments and possibilities?
Takakura：Contributing to mobility is one large area that Sony is currently tackling, and while it is rather difficult challenge, there are also great opportunities. We believe that vehicles will be moving sensors in the future. We think about what kind of value we can create with these connected sensors. In this rapidly changing social environment, the way that customers and markets see the mobility is changing, so we will need to consider how to expand our activities from a broader perspective.
Through collaboration with Sony Creative Center, we are promoting activities aimed at optimizing mobility by visualizing and analyzing various data.
2. Optimizing transportation
with demand prediction services
──How did you use AI technology and status data to develop this service?
Takahito Migita：We have status data on around 10,000 vehicles dispatched primarily in the Greater Tokyo Area by taxi companies affiliated with Minnano Taxi. First, we set about visualizing the data while accounting for inaccuracies and missing elements. As a result, we were able to find out how the levels and development patterns of demand differ based on the characteristics of each location, including those such as shopping centers, major stations, and residential areas. In addition, we also collected information that we felt was relevant to demand, such as that on events, the weather, and the running status of trains, and by analyzing this alongside the taxi status data, we considered what data was necessary for demand prediction and could become a feature—input data that is used during machine learning.
Before we properly got started with the analysis of the data, we also held discussions to find out how taxi drivers actually predict demand and think about a day's work. We then made use of the knowledge we had gained and data we had collected, and in collaboration with Sony's R&D Center, we started to develop a prediction system through machine learning. After testing various machine learning technologies such as decision tree models and deep learning, we selected the best method based on questions of cost and accuracy on which to build our system.
──During development, what points did you consider and design primarily for taxi drivers who will actually use the system?
Migita：As the users are taxi drivers, we of course focused on accurate demand prediction, but also put a lot of effort into making it easy to use and understand. During development, we also made sure to keep in mind that it was for those who drive vehicles for business, and so reduced the number of touch operations required to access information, and incorporated features such as sound notifications so that drivers need not take their eyes off the road as well as a way to switch off all notifications for when they have passengers.
We didn't just stop at making the application either. Since service began, we have continued to listen to drivers' opinions and requests, and have steadily been improving features and fleshing the system out. For example, in addition to supplying information about where most people are likely to be waiting for a taxi, we have also made it possible to find out about extra information that has an effect on demand for taxis, such as that on rainfall and public transport, nearby planned events, as well as details such as where long-distance passengers can be found, and how many vacant taxis are currently operating in the area.
The tablet application for drivers
──What kinds of areas can you contribute to with the demand prediction service?
Migita：First, it contributes to improving occupancy rates. Our original goal with this service was to improve these rates in order to boost revenue for taxi companies. Shortly after we introduced this service in November 2019, the numbers of passengers temporarily fell as a result of COVID-19, but drivers using the demand prediction service are now improving their occupancy rates.
Work efficiency of drivers and the optimization of energy usage are also the areas that need to be improved. For example, if the amount of business that used to take 300 km of driving can be done in 250 km, it will lead to direct reductions to working hours, and also help reduce the amount of fuel used and exhaust gas emitted. We believe that we can contribute towards social issues including overly long working hours and environmental problems.
──What feedback have your received from taxi companies using this service since it was introduced?
Migita：During our continued discussions with drivers, we have heard a number of comments such as "I check the demand prediction service when I can't find any customers, or when business isn't going so well at a particular time," "I check pick-up point information to decide where to go and where not to go," and "I have a standard routine for my day, so I tend to check the information on vacant taxis." It seems that they are using various different features of the service during their working days.
Of course, some users have also been asking for new features, or improvements that would make the app easier to use or more stable, and so we have been consistently providing updates and improving the system since it first launched.
The demand prediction service is being used for business from day to day
──What can you tell us about future developments and possibilities?
Migita：We hope to expand the service to other areas in addition to Tokyo, and gather more data and make it useful through analysis by increasing the number of partner companies,.
By developing demand prediction technology, movement efficiency will also improve. If we expand this technology to bicycle and car sharing, as well as public transport such as trains and buses, we are hoping that overall methods of transport can be optimized, and movement made more efficient. We believe that it could be applied not just to the transport of people, but also to that of goods.
