mwang7
October 9, 2024, 1:20pm
1
Hi, I’m trying to build an app that record the desktop and later the audio and stream it over the network.
Basically you can break it down like so :
Grab the desktop frames with DXGI desktop duplication API in BGRA format
Encode with NvEnc to H264
Packetize into RTP packets with FFmpeg
Send to the network
I currently am able to stream with a good image quality.
The issue is that there is a time delay issue and I believe it could come from the encoder’s parameters related to timestamps
For test purpuse here i stream locally using VLC :
As you can see in there, on the difference between the desktop and the stream is significant.
I tried to look into the documentation and it’s saying this :
I tried to look into my code and I was using the default settings set by the sample, modified a few parameters like GOP length. enablePTD in NV_ENC_INITIALIZE_PARAMS and , change the delay to 1.5 sec and defining all the frames as IDR (stilll too much)
Currently i’m having some confusions :
Is the parameter enablePTD a way to not manage ourselves if the frame is a IDR, P-frame or I-frame and let the NVENCODE API decide ?
If i’m setting enablePTD to true, the stream is slow and end up having a 5-10-15 sec delay over the time
end up creating a delay too but since it tries to recaliber itself it’s less but still too much (~3 sec)
mwang7
October 9, 2024, 2:16pm
2
Also tried to set the IDR and I frames and P frames manually just to test it out,
if (m_iToSend == 0)
{
picParams.pictureType = NV_ENC_PIC_TYPE_IDR; // Start with IDR
std::cout << "IDR" << std::endl;
}
else if (m_iToSend % 30 == 0) // every 30 frames, use I frame
{
picParams.pictureType = NV_ENC_PIC_TYPE_I; // Use I-frame
}
else
{
picParams.pictureType = NV_ENC_PIC_TYPE_P; // Use P-frame for others
}
But it streams a black screen on VLC and it outputs this error on ffplay without displaying anything :
./ffplay.exe -protocol_whitelist file,crypto,data,rtp,udp -loglevel debug -i stream.sdp
ffplay version 2024-06-27-git-9a3bc59a38-essentials_build-www.gyan.dev Copyright (c) 2003-2024 the F
mwang7
October 10, 2024, 1:41pm
3
Ok found the reason why the delay, i was from another part, where in FFmpeg I packetize into rtp packets I’ve set 30 fps in initialization of FFmpeg whereas i’ve set 60 fps in nvEnc initialization.
