(Photo by Jaque Silva/NurPhoto via Getty Images)
Not content with just 200,000 GPUs, Elon Musk’s xAI startup plans to expand its Colossus supercomputer in Memphis, Tennessee, to one day house at least a million GPUs.
The announcement was made during a luncheon hosted by the Greater Memphis Chamber, which helped get the xAI facility built.
"The company is setting the stage for Memphis to become the global epicenter of artificial intelligence,” the chamber said. "The expansion, already underway, will incorporate a minimum of one million Graphics Processing Units (GPUs), marking the largest capital investment in the region’s history."
The Chamber didn't provide a timeframe, but xAI will likely buy the GPUs from Nvidia, which already supplied 100,000 H100 cards to the first stage of Colossus this summer. Normally, companies take years to build a supercomputer, but Musk’s team assembled the GPUs into a working supercomputer in about four months.
The company has since expanded Colossus to 200,000 Nvidia H100 and H200 GPUs with the goal of reaching 300,000 GPUs by next summer. The announcement underscores the lengths Musk is taking to create cutting-edge AI programs to beat the competition, including OpenAI.
The Grok chatbot came out of xAI; it can also generate images and program computer code. While Grok is mostly limited to X Premium subscribers, xAI is preparing to release a standalone consumer app to rival OpenAI's ChatGPT, according to the Wall Street Journal, which says Musk has an “obsessive focus on building bigger data centers faster than his competitors.”
Others—including Meta, OpenAI, and Microsoft—have also acquired huge quantities of Nvidia’s latest GPUs. In January, Meta CEO Mark Zuckerberg projected his company would have “around 600,000 H100 equivalents of compute” by the end of this year.
In the meantime, the Greater Memphis Chamber noted that Nvidia, Dell, and Supermicro Computer—which all helped build Colossus—plan to establish operations in the city. However, the rise of Colossus has also called for investments in infrastructure to power and cool the supercomputer, which has sparked environmental concerns.