Japan’s king of cute wants to develop a lovable A.I. you can’t live without

Whether it’s robots or smartphones, AI or premium audio products, Japan has always been at the forefront of any conversation about technology. We recently spent several weeks in Tokyo discovering not only what some of the biggest names in new tech are creating, but also taking advantage of the exciting location to test out the best smartphone cameras, and discover the charm of its popular tech-tourism destinations. Make sure to check out other entries in our series “Modern Japan.”

In fall 2011, Line found itself in exactly the right place at the right time. The Apple iPhone 4S launched for the first time on all the major carriers in Japan, and exciting Android smartphones were arriving everyday to challenge it. Smartphones were becoming hugely popular, and although Line was a fledgling company — having released a dozen other apps with limited success — the smartphone-adopting population wanted a messaging app. Line was it, and in just three months, it caught on in a very big way.

Japan — just like the rest of the world — is on the cusp of another technical revolution, this time involving artificial intelligence.  Over the past seven months, Line has developed its own AI called Clova, and it may be poised to have the same impact on AI adoption in Japan that it did on messaging six years ago. While other companies focus on using AI in our homes as a voice-activated controller or assistant, Line’s vision is to create AI that also fits into our lives as an everyday companion.

We visited Line’s new offices in Tokyo to learn more about its AI ambitions, and understand the impact it could have on how we view and interact with artificially intelligent systems.

What’s Line? 

Line as a messaging app isn’t well known outside of its four biggest markets, Japan, Thailand, Indonesia, and Taiwan. But where Line is used, it’s used a lot. Its name has organically become a verb. In the same way we Google something on the web, to “Line a friend” is a common phrase saying you’ll send them a message.

If you don’t know the messaging app, you may know Line Friends, the company’s cute characters. Brown the bear, Sally the duck, and Cony the rabbit are the best known, but there are many more. Each has its own distinctive personality and interweaving storyline, and this is important as it will become relevant later on. If you live in New York City, the Line Friends store is the place to be introduced to them all. It opened recently, and attracted 300,000 visitors in its first week. Line is also famous for pioneering in-app sharable stickers, which have since been adopted by Facebook Messenger, Google Allo, WeChat, and others.

Line is widely referenced in Japanese popular culture. It was the app used by the two main characters in “Your Name,” the anime movie that received international acclaim this year. It’s a common sight in anime and manga generally, along with appearing in mainstream television shows and even song lyrics.

Line and artificial intelligence

While its ubiquity in Japan and other Asian countries is clear, getting a large number of people to use it elsewhere is a challenge. But breaking into major new countries isn’t Line’s current focus, according to CEO Takeshi Idezawa. Instead, it’s hoping to once again get the timing right for the next major technology breakthrough.

“Rather than concentrating on geographical expansion at the moment, we are thinking about artificial intelligence,” Idezawa told Digital Trends. “We’ve always been mindful about what’s coming next, and we think the next big movement will be AI.”

This thinking led to the creation of Clova, Line’s own artificially intelligent assistant, and the Wave speaker on which Clova operates. Beginning just seven months ago, Clova has been built from the ground up, including the software, hardware, and command structure, creating the world’s first Japanese speaking AI. Line has big plans for Clova, and they aren’t restricted to the AI simply living on a basic speaker, turning the lights on or off, or checking the weather forecast.

Creating AI from the ground up

Most AI systems in use today speak English, or started out that way. Clova started life speaking Japanese, which presented some very unique challenges. Line’s Chief Strategy and Marketing Officer Jun Masuda told us about them.

”If we could have bought an existing AI base, that would have been nice,” he laughed. “But none were available, so we built everything.”

Describing it as a, “difficult mission,” he explained the main components of AI — voice recognition, language comprehension, and voice synthesis  — are fundamentally different when the language isn’t English.

Andy Boxall/Digital Trends

“In English the verb comes first to explain what the command is, but in Japanese, the verb comes at the end,” he said. “So the command processing is completely different.”

This means an English assistant knows instantly what skill it will need. Say “Play me some music,” for example, and the AI knows immediately it’ll need to use the audio player, and listens out for the desired artist or track. In Japanese, the structure is reversed, meaning the artist and song would come before the command to play. Therefore a Japanese AI can only narrow down the options until the end of the sentence, as the question could be a request for information about an artist, rather than to play a song.

How does this affect the way Clova listens for commands? The wake-up word for Clova is simply its name, and once it’s listening, Clova will make assumptions about the forthcoming command based on what you say. It will also learn by previous commands. For example, if you repeatedly ask to hear songs by a particular artist, Clova will prioritize that command for the future to improve speed and quality. But it gets harder.

“There are many different ways to express yourself in Japanese,” Masuda continued. “We have regular characters, along with those for expressing English words in Japanese, and also kanji, which are adapted from Chinese characters. All have to be processed by the AI for synthesis and understanding.”

Kanji poses a particular problem, as each character can have multiple meanings, and even people get confused over how they can be read. To build a functioning AI in less than a year, with challenges like these, seems like a monumental achievement.

A partner, not an employee

An AI assistant that can verbally interact with you is only the start. How do you get people to actually talk to it? After all, most people are used to touch interfaces, and are nervous of talking to devices. Masuda believes voice control will soon become the norm.

“It’s the most natural way to give a command,” he said. “Over the next 10 years,l we will enter a much more convenient era, when the relationship between people and computers will become much more natural. The hardware we are working towards is something that blends into people’s daily lives. There are two directions AI is moving in. One is like a remote control, and the other is more like a partner, one that converses with you.”

Takeshi Idezawa, CEO at Line Corporation

Line wants people to use Clova in all situations. It’s designed to have continuous, natural conversations, so there’s no need to say “Clova” before every interaction; but this also means Clova needs character and charm. If there’s one thing we already know about Line, due to Brown and the gang, it’s that it fully understands how to create lovable and relatable characters that people adore. When Clova was being created, the team wrote an entire profile, with everything from where it was born, its strongest skills, music it likes, and an educational background, to personality traits and how it interacts with friends. Line already knows who Clova is.

For Line, if a situation arises where there are two choices, the one people are most likely to fall in love with is chosen. When describing adorable, interesting products, Masuda referenced Sony’s Aibo robotic dog as an example. The normal-looking Wave speaker is the beginning, and the next Clova products will look like Line’s famous Brown and Sally characters, gradually making Clova more relatable.

However, it’s Line’s partnership with Gatebox where things take a sci-fi turn. Gatebox has developed a holographic-style AI character to make living with a digital creation a reality. By bringing these two systems together, Line hopes to achieve wider acceptance for digital, artificially intelligent creations.

“Others in the industry are creating a general, more scalable assistant, and so it’s kept plain,” he said. “But we want to use characters to build people’s affection towards the device. It’ll take longer, but it ties in with our company mission statement of, ‘closing the distance.’ It’s not enough just to connect people.”

Line has built Clova’s architecture in a way that allows other character types to sit on the top, and both software and hardware development kits are coming. A new generation of Gatebox digital companions, or even robotic companions like Aibo, are a tantalizingly real possibility. Masuda believes the creepiness factor of something like Gatebox will take several years to pass, but he considers us living with AI in this way inevitable, particularly in Japan where characters, from beloved local mascots to virtual popstar Hatsune Miku, are generally accepted.

“It’s not just the technology that’s important,” Masuda said. “The key is how that new trend can be introduced into people’s lives, and what kind of user experience is needed to do that.”

Timing is everything

CEO Idezawa knows Line is up against some serious competition in AI, not least because in October, Google announced Google Home for Japan. Line was aware of Google’s intention to launch a Japanese version of Assistant, and worked hard to release Clova before it. Idezawa believes in the importance of great timing, and there’s no doubt Line got it exactly right in 2011. Can this be repeated with Clova? The direction it’s taking is one few others are talking about, potentially giving it a crucial head start in what’s becoming the biggest, and most exciting tech trends we’ve seen in years.

Japan is Line’s priority for Clova, followed by the other countries where it’s well-known; however Clova is being built with English language support in mind for the future. There’s no timeline for when this will happen though.