GeForce RTX 4090 (NVENC) does not work on OBS

Hi.

I have GeForce RTX 4090, and i want to use NVENC on OBS to record video or push flow, but it does not work and i do not know why.

I tried to upgrade the NVIDIA driver and the OBS’s version, even upgrade the system version (from Ubuntu 18.04 to 20.04 to 22.04), but none of it worked!

Most importantly, NVIDIA told me that it is supported for HEVC 8k.

Can someone tell me how I can make it work or why?

thanks a lot!

Hello @nora.liu and welcome to the NVIDIA developer forums.

Please share a bit more detail:

  • The exact error message and possibly the log from OBS
  • Which OBS version are you using?
  • Your exact system setup (CPU, RAM, etc.)
  • The output of nvidia-bug-report.sh attached to this thread.

Thanks!

Hello @MarkusHoHo and thank you for the quick reply.

Thank you!

What you marked in the log is not uncommon and no issue that would cause NVENC not being used by OBS.

One thing you should do is to update OBS to the latest version supporting Ubuntu, which should be v29.

Then make sure you really have NVENC as the encoder selected in the OBS settings under Output. It is under “Streaming” which might be confusing. Here is a screenshot:

If it still does not work, please attach the complete output of nvidia-bug-report.sh AND the latest log of OBS here. Even if you don’t think it contains anything useful.

Thanks.

Now i have update OBS to the newest version(29.1.3).

And then i find some problems, here is some screenshot:



I update the nvidia driver to 530.



Looking forward to your reply.
Thanks.

As I said before:

And it might very well be that you also need to install the recommended driver for RTX 4090, which is v535.104.05 if I am not mistaken.

Now i’ve update nvidia driver to v535.104.05 already.

Here is the log of OBS and output of nvidia-bug-report.sh.

2023-08-31 11-40-15.txt (18.1 KB)

nvidia-bug-report.log (1.6 MB)

Thanks.

Thank you!

On checking those files I don’t really see any obvious issues. Everything looks ok, both on OBS side and on NVIDIA driver side. I even see that the recording worked:

11:40:57.260: [FFmpeg NVENC encoder: 'advanced_video_recording'] settings:
11:40:57.260: 	encoder:      NVIDIA NVENC HEVC (FFmpeg)
11:40:57.260: 	rate_control: CBR
11:40:57.260: 	bitrate:      2500
11:40:57.260: 	cqp:          0
11:40:57.260: 	keyint:       250
11:40:57.260: 	preset:       p6
11:40:57.260: 	tuning:       hq
11:40:57.260: 	multipass:    qres
11:40:57.260: 	profile:      main
11:40:57.260: 	width:        7680
11:40:57.260: 	height:       4320
11:40:57.260: 	b-frames:     2
11:40:57.260: 	psycho-aq:    1
11:40:57.260: 	GPU:          0
11:40:57.260: 
11:40:58.080: libfdk_aac encoder created
11:40:58.080: libfdk_aac bitrate: 160, channels: 2
11:40:58.236: ==== Recording Start ===============================================
11:40:58.236: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/yy/2023-08-31 11-40-57.mkv'...
11:40:59.429: [FFmpeg NVENC encoder: 'advanced_video_recording'] Applying DTS value corrections
11:41:00.523: adding 106 milliseconds of audio buffering, total audio buffering is now 320 milliseconds (source: 媒体源)
11:41:00.523: 
11:41:15.894: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/yy/2023-08-31 11-40-57.mkv' stopped
11:41:15.894: Output 'adv_file_output': stopping
11:41:15.894: Output 'adv_file_output': Total frames output: 457
11:41:15.894: Output 'adv_file_output': Total drawn frames: 46 (530 attempted)
11:41:15.894: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 484 (91.3%)
11:41:15.894: ==== Recording Stop ================================================

So how do you see that something is NOT working? What happens that you determine there is a problem?

Thank you for your quick reply.

The most direct basis is that the enc (video encoder usage) in the instruction “nvidia-smi dmon” is always 0%, when I work with NVIDIA NVENC HEVC (FFmpeg) as the video encoder for encoding (live/recording) on OBS!

If the output is as expected, I would not worry about the nvidia-smi status. I just ran a test where I recorded a 2560x1440 desktop with HVEC on a 4070 and there I also have nvenc at 0%. But the recording worked flawlessly.

It might be that the load is too low to register, or nvidia-smi has a bug, which i will try to follow up on internally.

I understand what you mean, but the problem is that the reason why I pay attention to this is because my program currently only needs to be hardcoded(NVENC).

And then it happens that it cannot be hardcoded on 4090 (CPU resources skyrocket, GPU resources are not used), and in the process of troubleshooting, I found that 4090 performs the same on OBS.

In addition, I have found that when I set the OBS output resolution less than 4K and the video encoder to NVENC (input is 8K), it is indeed “too low to register” as you said.

But the test source and settings of all the information I replied to you earlier are based on 8K, so the probability will not be because of “too low to register”.

Finally, look forward to your follow-up reply.
Thanks a lot!

I’m starting to run out of ideas now. Your setup should be more than capable enough to encode an 8k video source if you are able to do the same with 4k or less. The software configuration looks fine, the driver installation is ok, no obvious problems.

Would it be possible for you to share your test source so we could try to reproduce what happens on your end?

If you’re referring to a video source, I tried compressing the video I used, but it was too big to upload, here’s its media information.

If it refers to my program, sorry, due to confidentiality, it cannot be provided. But what I can guarantee is that there is absolutely no problem with the program hardcoding, and it has been hardcoded successfully on RTX 3080, RTX 3080 Ti and RTX A5000.

To be honest, I don’t have any other ideas anymore, I hope you can help me solve this strange problem.
Thanks!

Thank you!

I try to have someone reproduce your setup and see if we can observe the same behaviour.

To clarify again, if you run OBS with that file as a video input and try to encode it, the CPU usage goes up, GPU is not used and you do not get any encoded output. Is that correct?

But what I can guarantee is that there is absolutely no problem with the program hardcoding, and it has been hardcoded successfully on RTX 3080, RTX 3080 Ti and RTX A5000.

What do you mean by “hardcoded”? Are you using the Video Codec SDK directly to use NVENC? Or some third party library like ffmpeg or gstreamer?

Thanks!

Thank you very much for your reply.
Perhaps I didn’t make it clear before, but let me fully describe the situation I encountered.

First of all, I have a program developed using NVENCODE API(called A), which can be successfully hardcoded on RTX 3080, RTX 3080 Ti and RTX A5000 (success here refers to the normal fluctuation of enc encoder usage around 50%), and can also be successfully hardcoded on OBS, of course, the data source here is all 8K30FPS.

Then when program A runs on the RTX 4090, the enc is 0%. The same is true for OBS, but the encoding output is available here, I think it is OBS internal judgment that the hard code has not taken effect (?) is automatically replaced with soft coding, so CPU usage will skyrocket.

Finally, I tried to hardcode directly using the FFmpeg command in the process of troubleshooting, the same other graphics cards work, only RTX 4090 does not work (of course, the performance of work here is also the enc encoder occupancy).

Again, expect you to reply.
Thanks!