How to use drmWaitVBlank for nvidia (linux)

Hello. After searching the internet for two days, I am close to giving up :(.
If anyone can point me to an example, preferably working code, that detects
VSYNC of a given monitor while using the nvidia driver…

I learned that nvidia has its own libdrm - although I can’t find it.
I found someone who was using fd = drmOpen("drm-nvdc", NULL); but
that gives “No such file or directory” (possibly because I’m using the
wrong libdrm.so).
I tried opening “/dev/dri/card0” directly, but then drmWaitVBlank gives
“Operation not supported”.

It should be possible; I found nvidia documentation for some drmWaitVBlank
that isn’t a no-op. It says that the second argument isn’t supported, but not what
to pass then (if I pass NULL it crashes).

I added a kernel parameter to enable modeset, not sure if that is needed.
But sudo cat /sys/module/nvidia_drm/parameters/modeset gives ‘Y’.

You do not want to know what I had to do to pull this out - but at some point had a huge program that managed to print the name that is available: “nvidia-drm”. Whereever I found “drm-nvdc”, apparently that was something else, or for some specialized hardware… I dunno.

This (using fd = drmOpen("nvidia-drm", NULL); works, but in the end more or less does the same as just opening “/dev/dri/card0”, so that I’m still stuck at drmWaitVBlank returning -1 and errno = 95.

Hi @Aleric !

On what platform are you working here? drm-nvdc was used in NVIDIA Drive OS version 5, refer to this documentation: NVIDIA DRIVE OS Linux API Reference: Direct Rendering Manager

Other than that VBLANK access is not exposed by NVIDIA drivers on Linux.

There are other synchronization mechanisms in different Graphics APIs that should be used instead, depending on your use case. Maybe you could explain a bit what you would like to achieve and someone here might be able to help with that?

Thanks!

I am extremely desperate by now :/. I have been doing nothing else for days and days - currently compiling/debugging the nvidia-dkms - I don’t understand why NVidia isn’t just supporting the ioctl’s for libdrm (aka drmWaitVBlank).

What I want is to run my vulkan engine in debug mode and get a signal / interrupt / callback, or even have a function that simply blocks, till the next VSYNC (for a given monitor). I intended to use that to debug my code because often the GPU is waiting for a VSYNC (and doing nothing). If I can visualize when the VSYNCs happen then I will be able to understand much better what is going on. It might also help to let the code make some decisions. It’s just better to know when these vblank’s occur (for debugging purposes).

I don’t need a better accuracy than 100 microseconds; and everything is running locally. I don’t need to support more than one GPU to do these tests either.

The only tool that shows me the VSYNCs is Nsight-systems - but I want to have the signals in my own program, not visible in a thirdparty tool (that lacks a LOT of other information that I want to see).

Please tell me how to get this signal out of the nvidia drivers into my application. I can even make custom changes to the driver if need be (I’m already doing that).

The reason that drmWaitVBlank returns 95 (Operation not supported) is because the driver isn’t calling drm_vblank_init; in nvidia-515.48.07/nvidia-drm/nvidia-drm-drv.c there is:

#if !defined(NV_DRM_CRTC_STATE_HAS_NO_VBLANK)
    drm_vblank_init(dev, dev->mode_config.num_crtc);
#endif

where NV_DRM_CRTC_STATE_HAS_NO_VBLANK is defined, and thus drm_vblank_init isn’t called. Why though? Having the member no_vblank in struct drm_crtc_state (in the kernel header drm/drm_crtc.h) just means that you can set it to true, which will cause the DRM atomic helpers to send out a fake VBLANK event during display updates. I am not really interested in fake vblanks? The only reason to set that flag would be if the driver is just not having access to the vblank interrupts.

I commented out those #if / #endif lines, and then drmWaitVBlank returns EINVALID :/.

Can you please point me to these “different Graphics APIs”? Is there something Nvidia specific exposed in one of the nvidia libraries? Please help, because I failed to find documentation online that can help me.

The OSS driver release by NVIDIA is still very new, that means not all possible functionality is completely exposed.
But even in future releases I do not think that we will open up IOCTL level access to our GPUs.

I don’t need a better accuracy than 100 microseconds

You surely mean milliseconds, right? 10000 FPS would be a very fast display indeed.

For debugging purposes my first suggestion would also have been NSight, correct.
But changing the driver beyond the kernel modules that are exposed through the OSS license is something you should avoid unless you want to sacrifice stability. And by it does go against the EULA as well.

In any case, right now VBLANK is not exposed on libdrm API level on desktop GPUs. You found the correct define, but manually disabling the #if will not magically make the driver send VBLANK events through this API.

The Graphics APIs I was talking about are Vulkan or OpenGL or DirectX.

An since you are already talking about Vulkan (for which we have also our own dedicated sub-forum) you could try to use Fences and the VK_EXT_display_control extension. But sadly I am not a Vulkan expert and can’t provide a good example.

I hope this helps!

Allowing developers to wait for a vblank is not the same as “open up IOCTL level access to our GPUs”. Also IOCTL is the interface that a user uses to communicate with kernel modules; you already implemented loads of (drm and other) ioctls to communicate with the driver.

No, I meant 100 microseconds. It would be 100% useless to get vblank events with a precision of 100 ms, which is many many frames long?!

If Vulkan had a way to get vblank events, then that is how I’d be able to get them as well; in other words then the Nvidia drivers WOULD expose them. But, as you say, they don’t. I really don’t care if I get them through DRM or otherwise, DRM is just the most obvious way - and I am stumped that you talk about licenses in regard to these events :/. Is it really that much of a threat to Nvidia to give developers vblank events??

I’ll look into VK_EXT_display_control thanks.

Thanks for being so patient with me.

Feel free to strike my second sentence above, I am clearly not up to date with our publicly available Linux driver offerings and will need to read up on what we do and do not support in the OSS repository.

The discussion of which features of our GPU are exposed through which APIs is too complex to discuss here and I would not be able to give you a satisfactory explanation, I am sorry. But of course it is not about feeling threatened by VBLANK misuse.

Coming back to the topic at hand I am now honestly curious why you need 100 microsecond accuracy. Maybe it is because I misunderstand the term VBLANK? The VBLANK period (time it takes to blank the physical screen) will of course be in the microsecond range. VBLANK events on the other hand depend on the refresh rate of the display, meaning for example even with the newest 500Hz displays that would be at 2ms intervals. The microsecond aspect only comes into play if you dig down into the VBLANK/VSYNC spec of a given resolution/displaymode combination and need to know exactly when VBLANK is signaled in relationship to the actual VSYNC interval.

If you only want to execute some code or do debug at a VBLANK event I don’t see how you would benefit from sub-millisecond accuracy.

On a related note I also reached out to some of our Linux experts to see if they have suggestions how you could hook into the VBLANK event. I will reply here in case I receive some input.

Thanks!

Thank you for your time and efforts to help me.

A frame takes -say- 16.67 ms. If there is a delay between a command buffer commit and the start of executing that command buffer by the the GPU (which is made visible by inserting certain instrumentation into the command buffer and calibrating the GPU clock against that of the CPU) and I want to understand if that has to do with when a vblank occurs, then I need to know when a vblank occurs with a precision of -say- 1% of the total frame time, aka 0.1667 microseconds. I think that is inaccurate enough to not have to worry about HOW I get the event, that’s why I mentioned it. If I needed nanosecond precision then that was a whole different story and would require a specialized implementation that only Nvidia devs could write (build into the driver). 100 microseconds however is, in my eyes, not very demanding. A precision of 10ms, however, would be utterly useless because that is comparable with the frame time itself and therefore I’d still have no idea where during a given frame the vblank happened.

Thanks!

Thank you for clarifying this! Now it makes sense.

I’ll check a bit more why drmWaitVBlank() is not supported on Desktop GPU Linux.

In the meantime one hint I received was to look into DRM_MODE_PAGE_FLIP_EVENT, but I am not certain it fulfills your requirements with respect to timing.

Good luck for now!