This was the most momentous PC announcement in decades
âAI PC.â
Youâve no doubt heard the term by now, and if youâre honest, itâs probably made your eyes roll. In a year when âAIâ is tacked on to every tech product imaginable, what have been called AI PCs so far just havenât felt worthy of the designation.
But Microsoftâs announcement today dares to take the concept of the AI PC seriously. Microsoft calls them âCopilot+ PCs,â a new breed of laptop that has been completely reengineered to support ARM processors for the purpose of running persistent AI models at all times. You might not be as enthusiastic about AI as Microsoft seems to be, but thereâs no question: this is one of the biggest PC announcements in decades.
New AI superpowers
A lot has been made of the presence of neural processing units (NPUs) in the latest generation of Windows laptops, but thereâs a problem: right now, they donât have anything to do outside of background blur in video calls. Part of the problem is a lack of performance in the current generation, yes, but Microsoft says itâs also due to how the entire system has been architected. At its core, Copilot+ resolves this problem by reengineering Windows 11 all around the experience of AI â specifically by running a series of always-on, on-device small language models in the background.
âWe had to make extensions in our driver compute model to be able to have the NPU show up as a first-class citizen, a true third processor inside the operating system,â said by Pavan Davuluri, the lead of the Windows and Devices team, to a group of media. âWe had to do work to be able to add new AI APIs in the platform. We had new OS primitives that are powered by always-on running models inside the device.â
To start off, Microsoft is offering two specific use cases of AI built right into Windows. The first feature is the more controversial one, which is called Recall. In a sense, itâs a redesigned view of search â if you want to think of it that way. Using the timeline you can look through everything youâve done on your computer, and using the PCâs small language models, can help you find things using natural language. That might be an item you mentioned in Discord chat or line of text in a Powerpoint presentation youâre working on.
âImagine now this new systems architecture weâre talking about can actually understand in real time everything youâre doing on the PC, that it can create a semantic index of whatâs happening,â Yusuf Mehdi, executive vice president at Microsoft, told us in the same briefing. âSuddenly now, your ability to not only find something, but to even recall it is now possible, and that becomes a superpower.â
To achieve this, of course, youâre giving enormous amounts of data to the AI in your computer. According to Microsoft, Recall is private, local, and secure, and guarantees that this information wonât be used to train AI models. Furthermore, you can go in and edit this available freely, blacklisting certain apps or websites as desired.
In fact, Microsoft says these PCs should be thought of more accurately as âa sensor for AIâ now. Thatâs exactly the kind of terminology that excites some and terrifies others. Still, thereâs an obvious privacy concern with the idea of your PC recording and knowing everything you do on your PC â yet Microsoft seems convinced that itâll be a hit once people try it.
The other big AI features revolve around some more familiar ideas in how we currently use AI â creation. As described by Mehdi, weâre already all using AI for text and image creation, all currently powered by the cloud. But there are some problems with this approach, according to Microsoft.
âNumber one, itâs very expensive. I can tell you that because we pay for all these GPUs here at Microsoft,â Mehdi says, laughing. âThe second thing is the latency. Sometimes itâs fast, but sometimes youâre waiting, youâre waiting. You go up to the server; the servers are full. Thatâs a problem.â
Faster and cheaper (for Microsoft) models are one thing. But Microsoft imagines these on-device AI models opening up entirely new experiences in the realm of AI creation. In particular, Microsoft previewed a brand-new Photos app, with DALL-E level image generation built right in, allowing you to create and edit your photos all with natural language. As Microsoft notes, that doesnât involve tokens or logins â itâs all completed on-device.
âItâs good, but imagine now you have local models, AI models, that are now increasingly powerful, and theyâre powered by very fast and performant NPUs, but theyâre on the local device,â Mehdi explains. âNow, that creation gets unlocked. And imagine you take advantage of what we will do with the PC, where we have pen and touch, and you can really get in it. Suddenly, the ability to create can maybe leap to a new level.â
That might sound vague, but we have to learn on what Microsoft has been developing. And as expected, weâre already seeing that laptop manufacturers such as Lenovo and HP are building their own AI software on top of Microsoftâs features. This is really only the beginning. Microsoft also showed off a feature in Paint called Cocreator, real-time translation, Copilot recommendations in settings, reply to message notifications with one click, and much more.
These are software experiences that just wouldnât be possible for non-Copilot+ laptops without tanking battery life. In fact, apart from a new full-screen version of Copilot, these are features that are exclusive to the Copilot+ device lineup. Thatâs why Qualcommâs ARM chips are key to making this whole transition really work. And for the first time, both Qualcomm and Microsoft appear to be holding up both sides of the deal.
ARM makes it all possible
As if the AI revolution wasnât enough, these Copilot+ laptops had an even larger challenge to overcome. Infamously, Microsoft has made multiple attempts to fully support Windows on ARM, whether it was the original Windows 8 transition or the Surface Pro X. But these new Qualcomm chips make the leap ahead in performance that was needed to convince both Microsoft and laptop manufacturers that itâs the real deal.
But as Microsoft has learned the hard way, sticking some ARM chips into an ecosystem that isnât ready for it only makes things worse. To make it work, Microsoft has had to redesign its entire stack to be ARM-capable.
âWe engineered this update of Windows 11 with the real focus on AI inference and taking advantage of the ARM64 instruction set at every layer of the operating system stack,â says Davuluri.
This is no minor update to Windows â everything has been redesigned from the ground up. Thereâs a new compiler, kernel, and schedulers â all tuned to the individual chip itself to optimize CPU performance. Thereâs even improved memory management to balance the extra load that always-on AI models add.
Beyond the hardware itself, thereâs been a Herculean effort to bring first-party and third-party apps up to date too. After all, all the performance in the world doesnât matter if the apps themselves arenât optimized for the hardware. Microsoft says it worked with over 300 software vendors (both consumer and commercial) to rebuild their apps natively to ARM.
âWeâve made tremendous progress in building that platform, so much so that I think when you look at our data today for Copilot+ PC customers, weâre fairly confident 90% of their application minutes of use will be native on ARM,â Davuluri said. âThat was a key goal for us, and I think weâre there.â
Microsoft points to plenty of important apps that have been compiled to run natively on ARM, including Zoom, Dropbox, Netflix, Lightroom, Fresca, CPU-Z, and Firefox â just to list a few. There are some notable apps missing, of course, whether itâs Adobe Acrobat, Discord, or Slack. For those missing apps, Microsoft has also built a new emulator for running x86 apps called Prism in Windows 11. The company claims that the efficiency of Prism is 20% gen-on-gen, resulting in performance in these apps equivalent to the most recent Surface Laptop 5.
âThat new Prism emulator, combined with the sheer improvement in raw CPU performance, takes us to a place where we have great app experiences, great experiences across the entire breadth of the Windows catalog, whether theyâre native or emulated for us,â Davuluri added. âThatâs something weâre pretty excited about.â
Of course, emulation doesnât work equally across applications, and all of this needs to be experienced firsthand, but this certainly seems to be akin to what Apple did with Rosetta 2 in its own transition to ARM.
Welcome to the Copilot+ era
While the transition to ARM has been many years in the making, the focus on AI shouldnât come as too much of a surprise. Over the past year, weâve witnessed Microsoft embrace AI with reckless abandon, along with many of the biggest players in the game. By 2024, itâs already become central to everything the company is doing moving forward. But the Copilot Plus PC program takes things to another level. Microsoft wants this to be the beginning of a reset on what the PC is, and Copilot Plus is a way of kicking things off.
The new software features are neat, but Copilot Plus PCs will need to meet a fairly tight list of hardware specifications, with a powerful NPU being the most significant one. All Copilot Plus PCs require an NPU capable of 40 TOPS, a standard that outpaces not only the current selection of Intel and AMD chips, but also Appleâs most recent M4. Theyâll also need 16GB of RAM and 256GB of storage, but the NPU is really the clincher here.
âWe have been working with the broad industry, silicon providers, [manufacturers], software developers, developers of all kinds, and weâve been on this journey,â Mehdi told us. âWeâve started with Qualcomm.â
Microsoft has been working with Qualcomm for years, but the Snapdragon X Elite is unlike any chip the company has made yet. Using a variety of AI benchmarks, Microsoft was quick to show off just how much AI performance these NPUs are capable of â specifically at very low wattages. Thatâs true in terms of raw CPU performance too.
âNumber one, they will be the fastest, most performant PCs on the market by a very large margin,â Mehdi claimed about Copilot+ PCs. âYou take the fastest, most performing PC out there, letâs say itâs a MacBook Air with an M3 processor. These PCs will outperform that by 50% on the Cinebench benchmark.â
We got to see some of that performance firsthand at the Microsoft labs, proving that these benchmarks are for real. Testing the new Surface Laptop side by side against the MacBook Air, the Surface does indeed pass the MacBook Air in terms of multi-core performance across multiple benchmarks and applications.
Now, this first wave of Copilot Plus devices will all be built on the same Snapdragon X Elite chips. In the future, laptops with other chips may also qualify, such as the devices launching with Intel Lunar Lake. But Qualcomm has an exclusivity on it for now â at least when it comes to laptops you can actually go out and buy.
All of that adds up to something akin to an M1 moment for the Windows platform. We donât know for sure that Microsoftâs attempt will stick â and with an ecosystem this broad, itâs bound to be messier than with Apple. The adoption of AI at this level certainly throws a wrench into things, and how the public reacts feels like a moving target.
Iâll be the first to admit that it felt like Microsoft needed a miracle to to embrace both AI and ARM in one fell swoop.. Itâs been a long time since weâve seen Microsoft act this boldly, stepping up to lead the entire ecosystem toward a common goal. And from the looks of it so far, Microsoft just may have pulled off the impossible.
So, yes, with Copilot+, the term âAI PCâ is no longer just marketing jargon. If the phrase âAI PCâ has ever been meaningful, Copilot+ PCs will have certainly earned the badge. Whether thatâs a net positive in your book, though, Iâll leave for you to decide.