(Credit: Getty Images)
LISBON—A company that defines its mission as ensuring that “everyone and everything can be intelligently connected” is not an obvious suspect to support moving computing tasks offline.
But Qualcomm Chief Marketing Officer Don McGuire used a talk at the Web Summit conference here to tout the company's efforts in on-device AI. "A lot of this can happen on-device,” he said. “It doesn't have to go to the cloud."
McGuire expanded on that in an interview with PCMag, outlining concerns with cloud-based AI services (see, for example, OpenAI’s ChatGPT) that go beyond a mobile device not having sufficient bandwidth.
“There are privacy, IP [intellectual property], safety and security issues,” he said. “There are sustainability issues with regards to power and water consumption at the data center level.”
Plus cost issues, McGuire added: “I mean, an AI prompt on ChatGPT is seven to 10 times more expensive than a Google search.”
The new part of Qualcomm’s on-device AI pitch—as seen in nods to the concept during last month’s unveiling of the Snapdragon 8 Gen 3 smartphone chipset and the Snapdragon X Elite laptop processor—is how local computing resources can now field not just focused tasks like "computational photography" but also run large language models, should smartphone vendors build apps on those frameworks.
(I might as well note that I used a competing example of on-device AI, Google’s Live Transcribe, to capture McGuire’s remarks.)
“What we're doing now is putting an emphasis on the fact that our platforms could support large language models up to 10, 13, 15, 30 billion parameters,” he said. In comparison, the LLaMA model Meta introduced in February tops out at 65 billion input parameters, and OpenAI’s GPT-4 may hit 100 trillion.
McGuire looped back to the privacy advantages of on-device AI—something Apple and, more recently, Google have emphasized—by recalling his thoughts while watching a Taylor Swift documentary with his daughter and seeing how the singer-songwriter dictates song ideas into her phone. “Imagine if somehow that got into the public cloud through a ChatGPT sort of thing,” he said. “All of a sudden, these lyrics that she's developing are now in some algorithm on some large language model.”
Qualcomm’s concept of non-cloud AI also covers “on-prem” services in which an organization hosts its own AI resources, so the corporate data still doesn’t wind up in another company’s large language model.
On-device AI models can, however, leave users with a longer feedback loop because user reports of incorrect results don’t have a direct path back to the AI mothership. McGuire said Qualcomm relies on “federated learning” models, in which software on a device uploads an anonymized, abstracted report of glitches, but did not get into details.
Acknowledging that he’s seen AI chatbots serve up wildly incorrect versions of his own life story (“I’ve died, apparently already a long time ago”), McGuire said gen AI has “been a bit of a Wild, Wild West, and I think that has to change.”
But that doesn’t mean he favors government regulation of the sector. “You know, ‘regulation’ isn't a very interesting word,” he said. “I think there needs to be governance.”
Asked to expand on that, McGuire added that legislative and rulemaking processes run too slow. He said tech companies should move ahead while striving to avoid the errors of social-media platforms that aggressively experimented with boosting engagement without considering their side effects.
“If we can learn anything from that model, what can we do better?” he asked. “I think that'd be a good place to start.”
But how much is the word “if” doing in that sentence? “I think it depends—on the who and the what,” he said: ”But I'm an optimist, so I think we can collectively figure it out.”
(Disclosure: I moderated three panels here, and the organizers are covering airfare and lodging.)