[DLSS] Suggestion: Enhancing Frame Generation for Story-Driven Games via Pre-Generated Scene References

Gaming and Visualization Technologies General Topics & Other SDKs
, , ,
1

Hello everyone,

I’m not a developer, but I’ve been following DLSS advancements closely. I have a suggestion that I believe could improve Frame Generation specifically for story-driven single-player games.

The Core Idea:
Instead of generating every frame from scratch, what if we pre-generate “reference frames” for key scenes in a mission (e.g., flying a plane, parachuting, driving, entering a building) and store them in the game files?

During gameplay:

  1. The game loads these reference frames into VRAM (2-4 GB, which is feasible on modern GPUs).
  2. A lightweight AI model adapts these frames based on the player’s actual position, camera angle, and actions.
  3. This would reduce the computational load of full frame generation, lower latency, and save VRAM.

Why this could enhance DLSS:

  • Story missions are predictable; the AI can “know” what’s coming.
  • This approach wouldn’t replace DLSS but add a new mode optimized for slower, narrative-driven games.
  • It could allow higher frame generation quality on lower-end RTX cards (like my RTX 3050) without sacrificing performance.

I’m aware of challenges:

  • Storing reference frames would increase game file sizes.
  • The AI model would need to adapt to player choices (e.g., different hiding spots, timing).
  • This might not work for open-world or multiplayer games.

But for linear story games, I think this could be a valuable enhancement.

I’m not a developer, so I’d love to hear your thoughts. Is this technically feasible? What hurdles am I missing?

Thanks for reading!

2

To add more technical context on how this could work:

  1. Pre-production phase:

    • During game development, developers would use NVIDIA-provided tools to record and compress “reference frames” of key mission scenes (e.g., flying, parachuting, driving) from multiple camera angles and player positions.
    • These compressed reference frames are then packaged with the game files (similar to textures or audio assets).

  2. Runtime phase (during gameplay):

    • When a mission starts, the game loads the relevant reference frames into VRAM (estimated 2-4 GB, which is feasible on modern GPUs).
    • Instead of a heavy DLSS model generating every frame from scratch, a much lighter AI model performs “advanced motion interpolation” on the fly.
    • This lightweight model takes the closest reference frame to the player’s current view and adapts (warps) it to match the exact player position, camera angle, and actions.

  3. Player decision adaptation:

    • If the player makes a completely unexpected decision (e.g., lands in a different spot), the model switches to the nearest alternative reference frame and applies the same fast adaptation.
    • This would drastically reduce latency and power consumption compared to generating full frames from zero, making it ideal for story-driven games where scene paths are predictable.

  4. Benefits for all GPUs:

    • Lower-end GPUs (like RTX 3050) would benefit greatly by enabling high-quality frame generation with minimal performance cost.
    • High-end GPUs (like RTX 5090) would benefit as well, by freeing up Tensor Core resources for other tasks (e.g., higher ray tracing quality, better resolution) while maintaining ultra-smooth frame rates.
    • This approach could potentially reduce power consumption and heat generation across all GPU tiers.
    • It could be offered as an optional “Story Mode” enhancement within DLSS, not a replacement.

I hope this clarifies the technical direction. I’d love to hear feedback from engineers on what challenges they foresee.