From AI Companion to 3-Round Combat: Real-Time Battle System with CUDA + OpenGL on Jetson Orin Nano

From AI Companion to 3-Round Combat: Real-Time Battle System with CUDA + OpenGL on Jetson Orin Nano

Hello NVIDIA Developer Community,

I would like to share a new combat-system update for my real-time 3D AI companion project running on the NVIDIA Jetson Orin Nano Super.

The project originally focused on a persistent 3D AI character that could interact with the user in normal conversation mode.

The new version extends the same character into a complete real-time combat system.

Instead of treating conversation and gameplay as two independent applications, the same persistent 3D agent can transition from normal AI interaction into an AI sparring arena, complete several combat rounds, and then return to the companion environment.

Video Demonstration

The video demonstrates the new battle system, including:

  • Transition from AI companion mode into game mode
  • Player and Boss HP systems
  • Three combat rounds
  • Round-specific visual environments
  • Player weapon selection
  • Boss active attacks
  • Guard, parry, dodge and dash actions
  • Real-time score and combo tracking
  • GPU-intensive combat VFX
  • Round transitions
  • Match completion state

The video is intended as a technical demonstration of the new combat architecture rather than a formal performance benchmark.

Three-Phase Combat Architecture

The current battle is divided into three increasingly intense phases:

Round 1 β€” Awakening
Round 2 β€” Overdrive
Round 3 β€” Cataclysm

Each round has an independent combat state and can change:

  • Boss attack behavior
  • VFX intensity
  • Scene appearance
  • Attack timing
  • Player/Boss HP state
  • Combat feedback
  • Round transition logic

The simplified state flow is:

AI Companion Mode
β†’ Combat Request
β†’ Arena Initialization
β†’ Round 1: Awakening
β†’ Round 2: Overdrive
β†’ Round 3: Cataclysm
β†’ Match Complete
β†’ Return to Companion State

The main design goal is to keep the 3D AI character persistent while allowing the interaction mode to change significantly.

Player Combat System

The current player control system shown in the video supports several real-time actions.

Attack controls include:

LMB β†’ Bolt
RMB β†’ Blast
1 β†’ Rail
2 β†’ Blade
3 β†’ Drone

Defensive / movement controls include:

Q β†’ Guard
E β†’ Parry
Space β†’ Dodge
Shift β†’ Dash
WASD β†’ Movement

This gives the player separate offensive, defensive and movement states instead of using only a single attack animation.

Each action is processed through the combat-state system before triggering the corresponding graphics or gameplay response.

Boss Active Attack System

The Boss is no longer only a passive target.

The battle system can trigger active Boss attacks during a round.

For example, the video shows attacks such as:

Frost Lance Array

and later:

Absolute Zero Apocalypse

These attacks have their own visual state, timing and player-damage interaction.

This produces a more interesting workload than a one-way attack demo because both the player and Boss can generate real-time combat events.

Combat State Separation

One of the main architectural goals is to keep the combat state independent from the rendering implementation.

Conceptually:

Input / AI Decision
β†’ Combat Command
β†’ Validated Battle State
β†’ Attack / Defense State
β†’ CUDA VFX Update
β†’ OpenGL Rendering
β†’ HP / Score / Combo Update

The high-level game logic therefore decides WHAT should happen, while the real-time graphics system determines HOW that event is visualized.

This is similar to the architecture I use for the conversational AI component:

LLM / Agent
β†’ High-Level Intent

rather than allowing the language model to directly manipulate low-level rendering data.

Round-Based GPU Workload

An interesting part of this version is that the GPU workload changes substantially across the three rounds.

Round 1 is relatively lightweight.

Round 2 introduces stronger fire / energy effects.

Round 3 contains significantly more particles, transparent geometry and large screen-space effects.

This makes the three-round structure useful for performance profiling because the same scene can produce several very different GPU workloads.

A simplified workload progression is:

Awakening
β†’ moderate VFX

Overdrive
β†’ larger attack geometry
β†’ more emissive / transparent effects

Cataclysm
β†’ large multi-layer VFX
β†’ high screen coverage
β†’ heavier fragment workload

Transparent VFX and GPU Cost

The later combat stages contain many overlapping transparent and emissive effects.

For example, during the Cataclysm phase, several layers can be visible simultaneously:

  • Large transparent geometry
  • Energy rings
  • Particle systems
  • Screen-space impact effects
  • High-intensity emissive effects
  • Bloom

This can create substantial fragment overdraw.

Based on previous profiling of my Jetson graphics workloads, I have observed GR3D utilization approaching 98–99% during the heaviest VFX stages.

Therefore, I am increasingly treating combat graphics as a GPU-budget problem rather than simply adding more effects.

Adaptive Combat VFX

One direction I am investigating is dynamically changing VFX quality according to combat intensity or GPU headroom.

For example:

Normal GPU load
β†’ full particle density
β†’ full-resolution effects
β†’ all secondary VFX

High GPU load
β†’ preserve primary attack effect
β†’ reduce secondary particles
β†’ reduce transparent layers
β†’ lower selected post-processing resolution

The goal would be to preserve the visual identity of each attack while avoiding unnecessary GPU workload.

AI Companion + Combat Integration

The longer-term goal of the project is not only to create a standalone fighting game.

The same 3D character acts as both:

AI companion
and
interactive game character.

This allows a workflow such as:

Conversation
β†’ User requests a battle
β†’ Combat mode begins
β†’ Battle state evolves
β†’ Match finishes
β†’ Conversation resumes

The 3D character therefore becomes a persistent interface rather than a separate chatbot placed next to a game.

Current Development Focus

My current work is focused on:

  • AI companion / combat state transitions
  • Multi-round battle logic
  • Boss active attacks
  • Player weapon state
  • Defensive actions
  • CUDA-generated combat VFX
  • OpenGL rendering
  • Transparent overdraw
  • Frame-time stability
  • GPU workload balancing
  • Returning cleanly from combat into normal AI interaction

Questions for the Community

I would appreciate feedback on several technical points:

  1. For a real-time battle system where each round produces a different GPU workload, would profiling each round independently be more useful than measuring only the average application FPS?

  2. When GR3D utilization approaches 100% during the heaviest combat effects, is adaptive VFX quality generally a reasonable strategy on Jetson?

  3. For large transparent combat effects, which would normally be the first optimization target: particle density, screen coverage, bloom/post-processing, or the number of overlapping transparent layers?

  4. Would rendering selected large VFX into a lower-resolution framebuffer and compositing them into the main scene be an appropriate optimization?

  5. For the transition between normal AI conversation and real-time combat, is keeping the LLM/agent layer decoupled from the high-frequency combat/rendering loop still the preferred architecture?

  6. If LLM inference occasionally overlaps with GPU-intensive combat rendering, would Nsight Systems be the recommended tool for identifying GPU contention between the two workloads?

  7. Would it be useful to prepare a minimal reproducible sample containing only one combat round, one Boss attack and one player attack for profiling?

Thank you for reading.

I am especially interested in the architecture of persistent 3D AI agents that can move between normal conversation and real-time interactive game states.

Any feedback regarding Jetson GPU workload management, CUDA/OpenGL integration, AI-agent scheduling or real-time combat rendering would be greatly appreciated.

1 Like