Nvidia Goes All-In on AI Large and Small, Led by RTX AI Laptops (2024)

Nvidia, the company behind GeForce GPUs and high-end AI processing hardware, pushed to combine those two worlds at Computex 2024. The chipmaker revealed new ways that GeForce graphics hardware can bring faster, better AI tools to gamers and professionals alike. Nvidia's plans start with leveraging the AI capabilities of its GeForce RTX GPUs (mobile and desktop) to enable a host of new automated, intelligent features.

New GeForce RTX AI Laptops

Nvidia's growing AI investment will first be seen in a slate of newly branded laptops announced at Computex. Nvidia specifically calls these systems "GeForce RTX AI Laptops" because they offer new AI features that run on the gaming GPUs' Tensor cores.

(Credit: Nvidia)

The announced models are Windows Copilot+PCs, but Nvidia's AI features bring an additional set of capabilities on top of that, which rely on the latest RTX GPUs to power more demanding applications. Nvidia was also clear that Windows' Copilot+ features are powered by neural processing units, or NPUs, baked into the latest mobile processors—not the Nvidia graphics hardware. That distinction is crucial because it lets Nvidia power what it calls "Heavy AI" tasks on top of the so-called "Light AI" applications that Microsoft is adding to Windows.

However, Microsoft and Nvidia also announced that the Windows Copilot runtime would soon support GPU acceleration for local small language models like onboard chat assistants and other locally powered AI uses. In the end, RTX AI will help power

The new models include the Asus TUF A14 and A16 gaming laptops, the latest Asus ROG Zephyrus G16, the Asus ProArt PX13 and PX16 creator laptops, and the MSI Stealth A16 Studio. Nvidia mentioned only those four specific models of new "RTX AI" laptops, but it also stated that more than 200 RTX AI-branded laptops will arrive from top brands, like Acer, Dell, Gigabyte, HP, Lenovo, LG, MSI, Razer, and Samsung.

All of these models are already Windows Copilot+PCs, utilizing their NPUs to power Microsoft's new Copilot AI features in Windows. But they also provide Nvidia GeForce RTX graphics to drive demanding games and creative applications.

That extra muscle from the GPU will pair with a broader app and driver support for AI libraries and software development kits (SDKs) for wide compatibility with the latest models and apps—not to mention hardware acceleration for generative AI apps, like Stable Diffusion image generation.

Specifically, Nvidia claims that the hardware in these laptops can generate up to 686 trillion operations per second (TOPS), putting the 45 TOPS of CPU-bound NPUs to shame and providing support for AI features in more than 500 games and apps. Nvidia also claims that, even compared with Apple's M3 Max processor and graphics, the new RTX-equipped laptops will deliver seven times faster performance in Stable Diffusion 1.5 image generation. These new laptops will also drive 10 times faster large language model (LLM) performance using Llama 2 7B at batch size 8, to be specific.

(Credit: Asus)

For example, look at the Asus TUF Gaming A14 (FA401) and the 16-inch TUF Gaming A16 (FA608). These ultraportable gaming machines combine AMD Ryzen processing, Nvidia dedicated graphics (up to an RTX 4070 laptop GPU), updated cooling, and upgradable storage and RAM. Using the AMD Ryzen processor, Asus claims that the NPU can generate up to 50 TOPS of AI performance. But using the Nvidia GeForce RTX 4070 GPU can generate up to 371 TOPS for AI acceleration of all sorts of AI applications, from gaming to locally run LLMs and image generators.

Asus has yet to announce pricing or availability for these models, but we expect them by the end of the year.

Mapping Nvidia's AI World, From Laptop to Cloud

AI is getting more complicated with AI features extending from web-based chatbots, like ChatGPT and Google Gemini, to on-device AI features, like Microsoft's new Windows Copilot+ features that run entirely on your laptop. Nvidia's been a player in that world for years, and the latest announcements from the GPU giant bring clarity to this more significant, more nuanced frontier of computing.

(Credit: Nvidia)

In a recent meeting with Nvidia reps, we were treated to an interesting explanation of the different realms of modern AI. Nvidia categorizes the hardware needed for AI use into three categories: "Light AI," "Heavy AI," and "Cloud-Scale AI."

At the smallest scale is your own computer. On-device AI has advantages like keeping data secure and private and being available to anyone with a capable device. In this case, that means anyone using a processor that has an NPU—like the NPUs that Windows Copilot runs on or the Neural Engine in Apple's M-series processors. That's ideal for some specific uses, but it's limited by the fact that NPUs can only generate 10-45 TOPS, the de facto standard for AI capability.

GPUs allow more powerful uses on personal devices, ramping up to 200-1,300 TOPS, hence the "Heavy AI" label. This lets individuals do a lot more with locally installed LLMs or image generators that run on their own machine, leveraging the powerful graphics processor and large VRAM to enable these more demanding tasks. These locally run tools are still private and secure but not as energy-efficient as the NPU-driven uses of "Light AI."

But the real power comes in on the cloud, where hardware is scalable, and you can generate TOPS higher in the thousands as needed. This is the model used by Open AI, Google, and smaller firms that offload AI computing to cloud servers. This category is exploding right now, with the most prominent companies investing in their own servers and hardware. Meanwhile, others are buying into the growing ecosystem of scalable cloud resources that apply to whatever app or model you need to run.

Nvidia sees a place for all of these uses, planting its flag as the leading provider of hardware for GPU and cloud-based applications. For consumers and tech enthusiasts, this means that AI tools are about to get a lot better.

Charting New Territory

Part of this effort involves Nvidia creating tools and apps that use GeForce gaming GPUs and RTX-accelerated software to make AI more accessible and helpful. While the company mentioned several applications and potential uses of these technologies, three stood out in particular.

Nvidia RTX AI Toolkit

Nvidia's RTX AI Toolkit helps fill the gap between smaller apps running at the device level and general-purpose AI apps that use cloud hardware and expensive pre-trained models.

(Credit: Nvidia)

The toolkit contains tools for customizing and optimizing AI models. It gives you more ways to tailor a custom LLM model for either on-device or cloud deployment, let it scale to different uses and hardware, and apply existing models to apps. All of this is also naturally optimized to run on RTX hardware.

(Credit: Nvidia)

Nvidia's RTX AI Toolkit seems cool for AI enthusiasts who want to tinker with LLMs and other AI models. However, it is even more useful for small businesses and freelance developers who want to leverage AI to improve their products.

Generative AI for Creators

Nvidia GPUs will soon provide RTX acceleration for generative AI apps, using the hardware to drive faster, better image creation in tools like Automatic 1111 and Comfy UI. The firm will also update its RTX Remix modding tools, which combine existing game mod tools with generative AI to allow rapid upscaling and reworking of game assets when making mods.

That means that, instead of painstakingly matching new textures to capture the look and feel of an older game, you can simply take a low-res texture image, describe what it is as a language prompt, and get a high-res version based on the original.

RTX hardware will also be used to enhance streamed video, with AI upscaling and image clean-up, all running on Nvidia RTX GPUs. This includes RTX Super Resolution, which makes for high-quality upscaling in apps like VLC. Nvidia has partnered with Black Magic and DaVinci Resolve to bring this same technology to those apps.

(Credit: Nvidia)

If you're interested in adding the tech to your own app, Nvidia has a new RTX SDK that lets you build the capability directly into apps. Developers' apps will look better on any system with Nvidia graphics.

G-Assist Lends a Hand

Finally, Nvidia showed a tech demo of an exciting new feature called G-Assist. This tool uses LLM technology to serve as an in-game assistant that can answer questions about game mechanics, story, and lore and generally takes the homework out of getting into a new game by making that information available as you play.

(Credit: Nvidia)

By showing a video of someone playing Ark: Survival Ascended, Nvidia demonstrated how you can ask questions about different game details and get answers pulled from official game materials, fan-made wikis, and other sources. Because the assistant can also view on-screen content, it can identify objects, give tips on gameplay, and otherwise provide help and information that would typically require leaving the game to research.

Another cool use shown in the demo was using G-Assist to optimize the gaming laptop system for better in-game performance, tweaking things like frame rates and fan settings to tailor the performance to your requests. This demo does not announce any new service (yet), but it is an exciting use case for personalized AI.

Between the new AI-enhanced RTX AI Laptops and the growing arsenal of AI tools and developer kits coming from Nvidia, two things seem clear: AI is growing to be a lot bigger this year, and Nvidia will make sure to be right in the middle of it.