Today, Microsoft announced its first World and Human Action Model, or WHAM, a generative AI capable of generating game visuals and controller actions. Nicknamed “Muse,” this AI was built in collaboration with Microsoft Research Game Intelligence and Teachable AI Experiences alongside the game studio Ninja Theory.

Muse is designed “to effectively support human creatives,” according to Microsoft. The AI was trained on Bleeding Edge, a multiplayer arena battler, but the intended applications go far beyond just creating an AI-controlled character (although that is one possible outcome.)

“What’s groundbreaking about Muse is its detailed understanding of the 3D game world, including game physics and how the game reacts to players’ controller actions. This allows the model to create consistent and diverse gameplay rendered by AI, demonstrating a major step toward generative AI models that can empower game creators,” says Fatima Kardar, Corporate VP for Gaming AI.

Credit: Microsoft Research Blog

The team has great expectations for this new AI, even though it’s still in a relatively early stage of development. Muse could make it possible to bring back older games that are no longer playable on modern hardware by optimizing the game — or potentially rebuilding it from the ground up. “To imagine that beloved games lost to time and hardware advancement could one day be played on any screen with Xbox is an exciting possibility for us,” Kardar said.

Taking that a step further, Kardar suggests Muse has the potential to create new content for existing games, and perhaps one day inject it on the fly. As AI models like Muse develop, they could have a massive impact on not only how games are made, but also how quickly.

Microsoft is touching on the kind of AI explored in popular culture, all the way from anime like Sword Art Online to popular novels like Ready Player One. And it isn’t so far away that you won’t be able to experience this for yourself. “We’ll create opportunities for people to participate in this exploration, starting with short interactive AI game experiences for you to try on CoPilot Labs very soon.”

Despite the potential, the use of AI raises concerns. The Muse team also addressed this, promising that it will “continue to be built on our commitment to Responsible AI” and that it would be guided by six principles: “fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.”