(Credit: Shutterstock / rafapress)
Google is sharing more information about what happened to make it pause Gemini’s image generation of people this week.
Earlier this week, Google Senior Director of Product Jack Krawczyk, who is overseeing the AI model’s development, posted on X that the Gemini team was working to adjust the model so it provided historically accurate results.
The post came after many users complained about Gemini creating historical images that depicted people of different races in photos where they likely would have not been present. For instance, Gemini offered a picture of a founding father who was African American and a Native American man and Indian woman as representative of an 1820s-era German couple
That fix turned out to be a bit more complicated than the team originally thought, and early Thursday morning Google disabled the ability to generate images of people with the tool.
In a blog post Friday, Google SVP Prabhakar Raghavan went into a bit more detail about what went wrong with Gemini. Specifically, Raghavan said that “our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range." He also said that “over time, the model became way more cautious than we intended, and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive.”
In other words, Gemini had been trained to show a variety of different races; however, it didn't know that there were times when that might not be appropriate. It also had a different interpretation of what might be inappropriate than was intended.
Ultimately, the model was led to “overcompensate in some cases, and be over-conservative in others, leading to images that were embarrassing and wrong," he said.
Raghavan says Google plans to make significant changes before turning it back on — an improvement process that will “include extensive testing.”