(Credit: Microsoft)
Copilot+ does not equal Copilot, but both are getting big upgrades today.
Copilot+ refers to the new class of PCs that sport an NPU. They’re available in Intel- and AMD-powered PCs as well as Qualcomm-powered Windows on Arm PCs, and today, Microsoft unveiled several new features for both classes of device. The just-released 24H2 version of Windows, aka the Windows 11 2024 Update, adds features for all Windows 11 users, but there are a few extra special ones just for Copilot+ PC owners.
The most interesting one is Click to Do (more on that in a moment), but there's also Super Resolution in the Photos app, Generative Fill and Erase in Paint, and more intelligent Windows search. Microsoft also announced security improvements for the much-derided Recall feature, which some testers have nevertheless found impressive.
Click to Do
(Credit: YouTube/Microsoft)The Click to Do feature for Copilot+ PCs reminds me somewhat of Photoshop’s recent Contextual Taskbar, which sees what you’re doing in the app and suggests one-click options that make sense for your current activity. Click to Do lets you perform other AI functions in Copilot+ PCs—like blurring the background in a photo, summarizing text, or performing a visual web search—with a single button press. You call up the feature by tapping the Copilot key, and then clicking on anything on the screen to get action suggestions.
Super Resolution
(Credit: Microsoft)This Copilot+ PC feature resembles those found in paid photo software like Adobe Lightroom and Topaz Photo AI. In the updated Photos app, you can take a low-resolution photo shot from years ago on a cheap phone or camera and make it look acceptable rather than something that makes you squint at those jagged edges on objects. This is similar to but separate from Copilot+ Automatic Super Resolution, which improves gaming frame rates.
Improved Windows Search
(Credit: Microsoft)With this update, you’ll be able to use real language in search rather than just matching characters in filenames. For example, you can enter “barbecue salad” to find relevant pictures or recipes. This works with local or OneDrive files and even in the Settings app’s search box.
Generative Erase and Fill in Paint
(Credit: Microsoft)Paint in Copilot+ PCs already let you use the Cocreate feature to create images based on your rough sketches, but now you can simply type in what you want the AI to generate from scratch, as shown with the lighthouse example above. You can now also paint over an object to remove it from the image and have the AI create content that fits the image.
In a briefing, Microsoft CVP Yusuf Mehdi also noted that features that take advantage of Copilot+ PCs’ NPU capabilities are coming to third-party software. A prime example is Algoriddim’s Djay Pro, an audio-mixing app available in the Microsoft Store for Windows apps. This app uses neural processing to convincingly blend, for example, the vocals from one track to the drumbeats from another via its Neural Mix feature.