(Credit: Microsoft)
Microsoft researchers and Xbox staffers have created a generative AI model that can simulate a playable video game.
The prototype program, called "Muse," can churn out the in-game visuals and corresponding actions based on the controller input. The 1.6 billion parameter Muse model was trained “on more than 1 billion images and controller actions" from a 2020 multiplayer title called Bleeding Edge, which comes from the Xbox studio Ninja Theory
“What’s groundbreaking about Muse is its detailed understanding of the 3D game world, including game physics and how the game reacts to players’ controller actions,” Microsoft’s Xbox team wrote in a blog post on Wednesday.
The company describes Muse as the first of a new class of AI programs called “World and Human Action Model.” The result can not only achieve “consistent and diverse gameplay sequences” but also ensure that the game state persists even as the player makes modifications to their character or to the game world, Microsoft’s team stated in a research paper.
Others have also shown that generative AI programs are capable of simulating classic games, including Doom and Super Mario Bros, but in a limited format. In Muse’s case, the program can only render gameplay sequences at a low 300-by-180 resolution over several minutes, resulting in some garbled graphics. The other limitation is that the generated gameplay isn’t always accurate to the player’s controller actions or to the game it was trained on.
Despite the limitations, Microsoft views Muse as a promising way to improve video game development and preservation. This includes using AI models to revive classic games no longer supported on current hardware. “You could imagine a world where, from gameplay data and video, that a model could learn old games and really make them portable to any platform,” said Xbox chief Phil Spencer in a video.
The company also sees potential to use AI models to help human software developers prototype game experiences on the fly. “We are already using Muse to develop a real-time playable AI model trained on other first-party games,” the Xbox team added, without offering specifics.
Still, Microsoft’s debut of Muse is bound to also spark fears and criticism about AI replacing human developers and destroying the creative spirit of video gaming. “Every time I think Xbox can’t get worse, Phil [Spencer] proves me wrong,” wrote one user on YouTube.
But in response to such concerns, Microsoft's announcement noted: "For Xbox, game creators will always be the center of our overall AI efforts. We believe there is space for traditional game development and future generative AI technologies that serve as an extension of creative work and offer novel experiences. As part of this, we have empowered creative leaders here at Xbox to decide on the use of generative AI."
In the meantime, other companies, including Netflix and EA, have also been embracing generative AI tools, seeing them as a way to streamline game development.