The State of the Art in Using LLMs in Games
To give you a sense of how much progress has been made in this field, let’s examine three tech demos and one published game that showcase the integration of LLMs in gaming.
🕵️♂️ Covert Protocol by NVIDIA and Inworld AI
Unveiled at GDC 2024, Covert Protocol is a tech demo that places you in the shoes of a private detective.
What’s interesting in this demo is the use of AI-powered NPCs that respond to your inquiries in real-time, influencing the narrative based on your interactions.
The demo is built on Unreal Engine 5, it leverages NVIDIA's Avatar Cloud Engine (ACE) and Inworld's AI to create lifelike character interactions.
Learn more here 👉 Inworld AI Blog
🤖 NEO NPCs by Ubisoft
Also at GDC 2024, Ubisoft introduced NEO NPCs, a prototype showcasing NPCs powered by generative AI.
These characters can perceive their environment, remember past interactions, and engage in meaningful conversations with players.
The idea here is to create more immersive and responsive game worlds where the player can have true interaction with NPCs.
Learn more here 👉 Inworld AI Blog
⚔️ Mecha BREAK Featuring NVIDIA's ACE
Mecha BREAK, an upcoming multiplayer mech battle game, integrates NVIDIA's ACE technology to bring AI-powered NPCs to life.
Players can interact with these characters using natural language, and the NPCs can recognize players and objects via webcam, thanks to GPT-4o integration. This innovation promises a more immersive and interactive gaming experience.
Learn more here 👉 NVIDIA Blog
🧛♂️ Suck Up! by Proxima Enterprises
Finally, Suck Up! is a published game where you play as a vampire attempting to gain entry into homes by convincing AI-powered NPCs to invite you in.
Each character is driven by generative AI, allowing for dynamic and unpredictable interactions.
Learn more here 👉 Suck Up! Official Website
Wait… Where Are the Agents?
After exploring these demos, you might be wondering: "These examples showcase the use of LLMs in games but they don't seem to involve Agents. So, what's the distinction, and what additional capabilities do Agents bring to the table?”
Don’t worry, it’s what we’re going to study in the next section.