Perspectives vol.2
What does AI make of a story taken out of its timeline?
In Perspectives, we visit people who are experts in their fields, interact with diverse ways of thinking, and learn through creation.
In this edition, we visited Yohei Sadoshima, CEO and editor at Cork, an agent for manga artists, novelists and other creators. He previously worked at the publisher Kodansha, where he was in charge of titles such as Dragon Zakura and Space Brothers. Seeing Sadoshima's interest in aibo, UX designer Junichiro Sakata used AI to analyze Investor Z, a manga about investing, and made the results into a movie. Here's why he analyzed the manga and what he learned from the results.
Learning only comes from experience
"Artificial intelligence reads the Investor Z Vol. 21" was an experiment where the latest volume of Investor Z was put into an AI available to the public. The findings of the analysis were then made into a movie. You can get a rough idea about the story in Vol. 21 by following the dialogues related to the protagonist Zaizen along spirals appearing on the screen. Among all the dialogues, AI considered a scene where Zaizen was stressed as lowest in relevance in the context of the frames and story before and after the scene. In this scene, Zaizen feels intense pressure about the future of the investment department he belongs to, even though he was not shown to be feeling much pressure until then.
With it being the first time a scene like this came up, AI was unable to understand why Zaizen was kneeling with his head in his hands and exclaiming phrases like "this is serious" or "what is this?!" If learning only comes from experience, as Sadoshima says, it follows that AI could not understand the scene because it has no experience of dealing with such pressure and no experience or foundation for analyzing such a scene. But for humans who read the manga, this is an emotional scene that has maximum impact. I think that one future challenge for AI is reducing this gap in understanding.
Uncovering Sadoshima's lines of thinking
The process of the analysis was as follows. First, I entered the manga into AI. I scanned all the pages of Investor Z Vol. 21 to create digital data, separated all the frames and created textual data from the dialogues. I marked each frame and fed them into AI in chronological order. I also reentered the information that Zaizen is the protagonist in the story. Then, AI sorted and categorized keywords, after which it went on to identify the relation between the protagonist Zaizen and the dialogues. I found that AI generally understood simple numerical data such as the keyword "Zaizen" appearing 42 times in the story, as well as the relationship between different scenes. However, AI gave an extremely low relevance score to the emotional scene in question.
Although I only analyzed Volume 21 this time, the results could have been different if I had entered all the volumes. It could also be interesting to identify Sadoshima's lines of thinking by entering other works like Dragon Zakura and Space Brothers. It may not exactly be reverse engineering, but I wondered whether we could use AI to capture the expertise of Sadoshima, the man behind many popular titles.
Aside from saying that learning only comes from experience, Sadoshima also talked about learning from fiction. He has accumulated vast knowledge since his young days and spent a lot of time thinking. This is what made Sadoshima so good at discerning whether something is interesting by intuition as well as reasoning. If we can use AI to gain insights into his works and how he creates them, we could use AI to support him when he's producing his next work. It would also be interesting to recreate Sadoshima's lines of thinking using AI, so that people around him can ask questions to or consult AI in place of the busy creator himself.
What kind of partner would AI make?
AI has recently started creating music and making paintings, but these creations aren't quite strong enough to stir up human emotions. This could be because of AI's gap in understanding mentioned earlier. Coming to that realization is the biggest lesson I've taken away from this experiment.
Existing AI is mostly used for raising efficiency in a fixed environment. For example, the AI for the game Go can constantly come up with optimal solutions and defeat humans. Yet, it still appears difficult for AI to leverage what it learns through Go in some other field.
The next step for AI is to get closer to becoming a partner like we see in sci-fi movies. When finding ways to bring AI into our everyday lives, instead of just using it to keep watch on things and make our lives more convenient, we should come up with ways to use it to generate emotional value in entertainment, hobbies, and other creative areas.
- Photo (left): Yohei Sadoshima
- CEO of Cork, Editor
- Photo (right): Junichiro Sakata
- UX Designer at Creative Center, Sony Corporation
Layout by Editing Department, AXIS
Text by Junya Hirokawa