(Image: PCMag)
It’s no secret that Microsoft has been leaning hard into the generative AI space with its ChatGPT-based Copilot app and service, with Microsoft CEO Satya Nadella going so far as calling Microsoft "the Copilot company.”
Just recently, the company unveiled new Copilot features for its Microsoft 365 suite of productivity applications. And today, the company unveiled the next stages in Copilot’s evolution, with a redesign and several features similar to those we saw OpenAI's reveal of ChatGPT-4o. They include more-realistic voice conversations and the ability to understand visual information and combine it with other input types, or what’s referred to as multimodal generative AI.
New features include Copilot Voice, Copilot Vision, Copilot Daily, Copilot Personalized Discover, and Think Deeper. The company also announced an early testing program called Copilot Labs, where select Copilot Pro subscribers will get to test new features before launch, including Copilot Vision and Think Deeper.
The update targets Copilot on mobile, Windows, the web, Bing search, and the Edge web browser. The new app and the non-Labs features start rolling out on Oct. 1.
In a video demo, Microsoft Corporate VP Yusuf Mehdi said: “We believe Copilot can go beyond answering basic questions and generating content, to offering more complete support for you and your tasks [and helping] match your energy, needs, and curiosity in the moment.”
The new features do seem genuinely helpful, even if they do further anthropomorphize tech.
1. Copilot Voice
Copilot’s voice interaction has been updated to be more natural sounding, and you have four new voice options. Reminiscent of OpenAI's Advanced Voice mode, Copilot Voice could probably pass many people’s personal Turing tests, so convincing is it that you’re talking with a human rather than to a machine. A neat feature here is that you can interrupt the AI voice, or even correct yourself when you misspeak without tripping up Copilot Voice.
2. Copilot Vision
(Credit: Microsoft)Copilot Vision is another piece of the multimodal puzzle, though it's only available to beta testers in the Copilot Labs program. It lets you share your camera with the AI to get answers based on what’s visible. But Copilot Vision in the Microsoft Edge web browser scans the pages you’re on and responds to verbal interactions with knowledge of the page’s contents. According to Mehdi, “When combined with the ability to just talk to it, Copilot Vision opens up a whole new way to use the web.” The example he showed was of someone trying to decide what movie to watch, and the AI filled him in on plots, actors, and review summaries. Mehdi noted that the feature is opt-in, clearly shows when it’s active, and interactions aren’t used for AI training.
3. Copilot Daily
(Credit: Microsoft)Copilot Daily is a personalized audio briefing of news, weather, and tips for the day, using the same lifelike voice options noted in the previous section. It seems like this could be preferable to only hearing about the day’s disasters and outrages in daily news broadcasts. (You can see how it looks in the web version of Copilot in the top image of this article.)
4. Think Deeper
(Credit: Microsoft)This new capability aims to solve complex problems with advanced reasoning. That includes math, logic, and coding problems. But it also includes things non-techies need help with, like planning multiple itineraries for a large family gathering. It's only in prerelease testing by a select group of paid Copilot Pro subscribers in the Copilot Labs program.
5. Copilot Personalized Discover
(Credit: Microsoft)This sleek new tool simply helps you get started with Copilot, suggesting things you can do with the AI. It bases its suggestions on your interactions with other Microsoft services (presumably including Bing and MSN) as well as your chat history with the AI. Your permission is required for this access.
One final new Copilot tidbit coming to the Edge web browser: You can now type @copilot in the address bar to summon the AI sidebar.