What you need to know about Sakana AI, the new startup from a transformer paper co-author

Founded by former Google AI engineers, the startup is inspired by schools of fish and bee hives to design collective artificial intelligence. …

Head over to our on-demand library to view sessions from VB Transform 2023. Register Here


The landscape of generative artificial intelligence (gen AI) is in a rapid state of evolution these days, not only with OpenAI, Meta, and Google competing directly on foundation models, but doing so while some of their biggest in-house talent leave to launch their own gen AI startups.

Last month saw one of the biggest defections: Llion Jones, one of the co-authors on the seminal 2017 research paper “Attention Is All You Need,” which kickstarted the generative AI revolution by developing the architecture of the transformers used in leading large language models (LLMs), announced he’d left Google to found a new startup.

This week, we learned just what he is working on: Sakana AI, a new AI company based out of Tokyo.

Joining him as a co-founder of Sakana is David Ha, previously the head of research at Stability AI and also a former Google researcher.

Event

VB Transform 2023 On-Demand

Did you miss a session from VB Transform 2023? Register to access the on-demand library for all of our featured sessions.

Register Now

Together at Sakana, they are pursuing a strikingly different model for AI: one of biomimicry, specifically looking to the collective intelligence in natural systems like schools of fish and bee hives, to design AI models that are flexible, reactive, and economically efficient.

At Sakana AI, Jones’ vision is to capitalize on this revolutionary technology to craft a trail-blazing generative AI model proficient in generating text, images, code, and multimedia content.

So far, Sakana is keeping its backers close to the vest, declining to comment on who is investing in the company to Reuters.

New tech inspired by nature

Sakana AI distinguishes itself with a novel approach that involves developing numerous smaller AI models that collaborate, much like a swarm, to deliver complex results.

This methodology challenges the dominant trend of constructing extensive AI systems.

Sakana AI is confident that its swarm-based approach, inspired by collective intelligence, can provide results on par with larger systems while being more economical and flexible.

“Ants move around and dynamically form a bridge by themselves, which might not be the strongest bridge, but they can do it right away and adapt to the environments,” Ha told Bloomberg in an interview. “I think this sort of adaptation is one of the very powerful concepts that we see in natural algorithms.”

The name itself was derived from a Japanese word for fish (さかな) and it’s supposed to elicit “a school of fish coming together and forming a coherent entity from simple rules”, the two founders told the Financial Times.

Sakana AI’s decision to establish its base in Tokyo is a calculated move. With its advanced technical infrastructure and highly educated talent pool, Tokyo is primed to nurture the growth of AI startups and attract international expertise. Sakana plans to exploit these benefits to bolster its research and development activities. Although, at this time, Sakana has yet to set up office space for its new headquarters.

Small and nimble start ups will be able to out-maneuver AI industry titans

The vision of Sakana AI transcends the creation of AI models. The co-founders underscore the shortcomings of contemporary AI systems that often end up inflexible. Instead, they propose AI models that embody principles of evolutionary computing, inspired by the adaptability of natural systems. This approach could potentially resolve issues related to cost and security in AI systems.

The co-founders mentioned to CNBC how they felt that Google’s focus on a single type of generative AI technology, large language models, was a mistake, “because that’s quite a restrictive framework,” said Jones.

“I would be surprised if language models were not part of the future,” said Ha to CNBC.

Jones’ and Ha’s transition from Google to Sakana AI is a noteworthy development in the AI sector, especially in the realm of generative AI.

“It’s just a side effect of big company-itis,” Jones said in the CNBC interview. “I think the bureaucracy had built to the point where I just felt like I couldn’t get anything done.”

Similarly, Ha also sees challenges facing the current gen AI leader, OpenAI. Ha said he views the use of the research he was part of at Google on a bigger scale as less than innovative and he voiced concern over the lack of sharing developments with the wider community.

By harnessing the swarm concept and evolutionary computing principles, Sakana AI aim to deliver flexible and cost-effective AI solutions. With Tokyo shaping up as a nucleus for AI innovation, Sakana AI’s presence in the city will contribute significantly to generative AI’s future evolution. The AI industry waits with bated breath to witness the transformation this collaboration proposes to usher in.

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.