Altera raises $9M to develop AI for digital humans

He added, “I started my group at MIT about two and a half years ago, where we specialized in building what we call multi-system, neural network models of the brain. If you really want to model the brain, it’s not just about modeling vision, or action, or thinking, or language. You really need to put the whole thing together, model everything. So when the language models become more available, we jumped right on it and started to build these language model based models of the brain.”

The focus has been how to turn this product into something that people can use, Yang said.

Altera’s mission is ambitious: to bridge the gap between artificial intelligence and human interaction by creating digital human beings. The brainchild of former MIT neuroscience professor Yang and his cofounders, Altera envisions AI agents with the capacity for empathy, friendship, and even fun, marking a significant departure from the current AI landscape. I interviewed Yang and cofounder Nico Christie for this story.

”The human brain is the only example of a high-functioning agent that we know of—able to learn and perform tasks autonomously for days and weeks,” said Yang. “The end state for AI products will not require back-and-forth conversation every 20 seconds. We need AI agents who can cooperate to solve complex tasks that may take time and who understand and care enough about us to solve problems we may not have articulated yet.”

Altera’s groundbreaking advancements have already begun to turn heads. In February 2024, the team unveiled an agent capable of playing Minecraft with others, effectively mimicking the experience of interacting with a human friend. This achievement not only showcases Altera’s technical prowess but also hints at the transformative potential of the technology across various industries.

The Altera Agent then achieved the famous “Voyager” paper benchmarks set by the Nvidia Research Lab by getting to the diamond pickaxe milestone with zero prompt iteration and a significantly cheaper model.

They started by creating a model of a digital human playing Minecraft. The Microsoft-owned game has a huge user base. And it’s an open ended game where you can spend a whole year if you want. So the team created an AI agent that could just play alongside a human player and be a companion.

“Minecraft allows us to really build agents that are sophisticated in the actions that they can take,” Yang said. “That really goes beyond just chat and walk and talk. Our long-term vision is to build digital humans that can live with us, that love us, just like friends would, that can grow alongside us as time passes. And, and we’re pretty serious. We mean to give machines the fundamental qualities that all pretty much all humans share, but are missing in current machines. Things like empathy.”

Investors, too, are captivated by Altera’s vision. Brian Cho, cofounder and general partner at Patron, said in a statement, “As an investor, it’s not often you come across a startup demo that fundamentally shifts your perspective on human-computer interaction. The talented team at Altera has developed an extraordinary AI agent tech that has the potential to go beyond its gaming origins and impact numerous industries. We are thrilled to be part of their journey from the outset.”

The significance of Altera’s work extends beyond gaming. As Accenture highlighted in its 2024 Trends Report, the future heralds the rise of entire agent ecosystems, with AI evolving from solitary-role assistance to autonomous decision-makers. Altera wants to be at the forefront of this evolution, poised to create digital humans that not only excel in capability but also possess autonomy and authenticity.

“There exists a massive opportunity to create AI companions that engage in all areas of our lives. However, today’s AI lacks critical traits like empathy, embodiment and personal goals, which prevent it from forming real, lasting connections with people,” said Aaron Sisto, partner at First Spark Ventures, in a statement. “Robert and the team at Altera are leveraging deep expertise in computational neuroscience and LLMs to build radically new types of AI agents that are fun, unique and persistent across platforms. We are thrilled to be a part of their journey.”

Yang is a computational neuroscientist and former tenure-track chaired assistant professor at the Massachusetts Institute of Technology, known for his research on deep neural network models of the brain. He and the company’s other three cofounders, CSO Andrew Ahn, CTO Shuying Luo and CBO Nico Christie, shut down their applied research lab at MIT to launch Altera.

“Robert shut down his lab where he was leading AI researchers dedicated towards helping machines start to empathize so that we could start this business,” said Christie, in an interview. “Three weeks after that, we raised out seed funding.”

Ahn holds an MIT math PhD, where he focused on theoretical deep learning. Luo spent eight years at Google; the last four were at Google Workspace AI, where he focused on building the first consumer features on early large language models. Christie is a previous entrepreneur whose first startup has scaled to millions in revenue; he dropped out of his MBA at MIT to launch Altera.

“The Altera team brings a rare combination of proprietary research capabilities, strong commercial instincts and fast product velocity to a massive opportunity space in AI agents,” said Jonathan Lai, general partner at A16z Games, in a statement. “We had the privilege to watch them build 24/7 during Speedrun and couldn’t be more excited to continue supporting them in their vision to build new kinds of digital human beings.”

With Altera leading the charge, the dawn of digital human beings appears closer than ever before. As they continue to push boundaries and redefine what’s possible in AI, the world eagerly anticipates the dawn of a new era in human-computer interaction.

“We were very inspired by the fact that we were doing cutting edge research and directly translating that into product, which is very unusual. For example, DeepMind never did it. We want to build a model of the brain and eventually model humans.”

He wants you to encounter a digital human based on a language model and say, “Wow, that’s just amazing,” he said.

Yang said he wants AI to really care about humans. Not a romantic love. But a love like how a dog would love an owner.

“We want them to have life experiences,” Yang said. “We want them to have fundamental drives. We want them to want to socialize.”

The team is pretty tight. Luo spent a lot of years at Google, researching the brain and other AI products. Luo is the CTO and she also happens to be married to Yang — for 12 years.

Why games?

I asked Christie why they’re focused on games first. They saw that with agent technology, the future is bright. But he said they wanted to put the tech right in the hands of regular people. And many of the team members had experience in games and even things like the Simulation Theory — where the belief is that we’re not real and we’re living in a simulation.

A debate on this topic about the Simulation Theory is how Yang met one of his cofounders, Christie.

“It’s hard to tell whether we are doing a simulation, but we can say for sure our agents are living in a simulation,” Yang said. “Once agents get smart enough to build their own simulation, and agents that live in Minecraft, then we’ll get to observe whether they wonder for themselves, whether they’re in a simulation.”

In gaming, the team wants to build an agent that can play a game like Minecraft and just be your friend. Or whatever game you want to play. You take a lonely single-player experience and make it more fun with an AI companion. It’s a persistent agent, Christie said.

“We can do it convincingly,” Yang said. “We don’t want to make an agent that is overpowered.”

Artificial life

The company has nine people. The name of the company isn’t just a nod to AI. It’s more about artificial life, which the company is trying to build. The question is how much intelligence does the AI need.

“We wanted to build a future where there are digital human beings,” he said. “When we say digital human beings, we believe there are fundamental human qualities that are missing from AI. We all have a very specific view for what is a digital human,” Yang said. “On the contrarian side, we think digital relationships should enhance and supplement human to human relationships, but never replace a human human one. I think the consequences of the paradigm shift we’re facing now are going to be far more profound than the ones we faced during the social era and during the internet (creation). And I think it’s important that we build a future we want to see by building it, and building it best and fastest.”

While Epic Games is working on MetaHumans, or those that look realistic. Altera is happy to focus on AI behavior.

“We chose gaming not only because it makes the most sense for our path forward. But it also is another way to differentiate some of the technical talent on the team and what we’re building,” Yang said. “We don’t really believe that our competitors are building what we’re building in the game space.”