Deepstream 6.2 , message payload issue

psharma3 · August 17, 2023, 9:43am

NVIDIA-SMI 525.105.17
Driver Version: 525.105.17
CUDA Version: 12.0
deepstream-6.2

I have observed when i m doing inferencing over two videos of same fps but of different duration ,that message payload for smaller duration video not coming after 1 st loop over , but for other video of higher duration its coming.

video_1 10 min fps - 25
video_2 60 min fps -25

why its happening like this , not getting msg payload for video_1 of short duration after 1 st loop over.

when i m changing fps of video, keeping different fps of both video
video_1 10 min fps - 25
video_2 60 min fps -50
in that case issue is not there for msg payload , i m getting msg payload for both after over of 1 st loop.

its seems like something going wrong with msg payload generarion we pass video of same fps for inferencing.

can you please let me know , why this issue happening

fanzh · August 18, 2023, 2:17am

can you elaborate on “that message payload for smaller duration video not coming after 1 st loop over”? which sample are you testing?

psharma3 · August 20, 2023, 8:48am

I m using Deepstream test 5 application

I’m passing file-loop: 1 in config file so as to run inferencing on video in loop.

I have two sources active
source -1 small duration video - 10 min
source-2 large duration video - 60 min

I mean to say when first loop over , means inferencing over small duration video overs
, after that i don’t see payload message of small duration video , atlhough inferencing going is going on as video is set in loop , but source 2 ,message payload is coming as size of video is big.

fanzh · August 20, 2023, 2:03pm

what do you mean about “payload message”? how to reproduce this issue? could you share the configuration file? Thanks!

psharma3 · August 21, 2023, 4:47am

below sink is msg broker sink , gives payloads through kafka

tests:
file-loop: 1

sink6:
enable: 1
type: 6
msg-conv-config: msgconv_config.txt
msg-conv-payload-type: 1
msg-conv-msg2p-new-api: 0
msg-conv-frame-interval: 50
msg-broker-proto-lib: /opt/nvidia/deepstream/deepstream-6.2/lib/libnvds_kafka_proto.so
msg-broker-conn-str: 192.168.0.118;9092
topic: quickstart-events

payload message means this one—
/*
The JSON structure of the frame
{
“version”: “4.0”,
“id”: “frame-id”,
“@timestamp”: “2018-04-11T04:59:59.828Z”,
“sensor”: “sensor-id”,
“objects”: [
“…object-1 attributes…”,
“…object-2 attributes…”,
“…object-3 attributes…”
]
}

fanzh · August 21, 2023, 9:44am

after inferencing over small duration video overs, can you see the source 1 video on the output video?
deepstream-app, nvinfer and nvmsgconv are opensource, can you narrow down this issue? you can add a sink probe or log in nvmsgconv to check if the source 1’s bbox was generated.

psharma3 · August 21, 2023, 4:40pm

on exploring i got to know , this issue is coming from some other reason

file - /opt/nvidia/deepstream/deepstream-6.2/sources/libs/nvmsgconv/nvmsgconv.cpp
in nvmsgconv.cpp i did some changes for custom schema payload type

else if (ctx->payloadType == NVDS_PAYLOAD_DEEPSTREAM_MINIMAL) {
message = generate_dsmeta_message_minimal (ctx->privData, frame_meta);
if (message) {
len = strlen (message);
// Remove ‘\0’ character at the end of string and just copy the content.
payload->payload = g_memdup (message, len);
payload->payloadSize = len;
g_free (message);
}
}
else if (ctx->payloadType == NVDS_PAYLOAD_DEEPSTREAM_PROTOBUF) {
message = generate_dsmeta_message_protobuf (ctx->privData, frame_meta, len);
if (message) {
payload->payload = g_memdup (message, len);
payload->payloadSize = len;
g_free (message);
}
}
else if (ctx->payloadType == NVDS_PAYLOAD_CUSTOM){

message = generate_dsmeta_message_minimal(ctx->privData, frame_meta);
if (message) {
  len = strlen (message);
  // Remove '\0' character at the end of string and just copy the content.
  payload->payload = g_memdup (message, len);
  payload->payloadSize = len;
  g_free (message);
}

}

inside NVDS_PAYLOAD_CUSTOM code section , i added generate generate_dsmeta_message_minimal(ctx->privData, frame_meta) function for the purpose of custom schema payload type.

on doing this , i m receving duplicate message like this
dulicate message with respect to frame id , id are getting repeat

{
“version” : “4.0”,
“id” : “1100”,
“@timestamp” : “2023-08-21T16:31:05.136Z”,
“sensorId” : “30BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=30BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E7BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E7BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“1|1256.98|176.68|1372.69|386.319|Jet Bridge Disonnected”
]
}
{
“version” : “4.0”,
“id” : “1150”,
“@timestamp” : “2023-08-21T16:31:07.129Z”,
“sensorId” : “30BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=30BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E7BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E7BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“8|1115.24|231.703|1172.59|249.674|Push Back Vehicle Connected”,
“1|1256.8|176.487|1373.07|386.105|Jet Bridge Disonnected”
]
}
{
“version” : “4.0”,
“id” : “1150”,
“@timestamp” : “2023-08-21T16:31:07.129Z”,
“sensorId” : “30BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=30BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E7BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E7BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“8|1115.24|231.703|1172.59|249.674|Push Back Vehicle Connected”,
“1|1256.8|176.487|1373.07|386.105|Jet Bridge Disonnected”
]
}
{
“version” : “4.0”,
“id” : “1150”,
“@timestamp” : “2023-08-21T16:31:07.130Z”,
“sensorId” : “42BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=42BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E9BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E9BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“7|789.78|501.91|1100.86|730.495|Push Back Vehicle Disconnected”,
“4|855.748|165.992|1111.61|540.454|airplane_front”,
“0|1244.37|154.76|1402.77|375.957|Jet Bridge Disonnected”
]
}
{
“version” : “4.0”,
“id” : “1150”,
“@timestamp” : “2023-08-21T16:31:07.130Z”,
“sensorId” : “42BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=42BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E9BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E9BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“7|789.78|501.91|1100.86|730.495|Push Back Vehicle Disconnected”,
“4|855.748|165.992|1111.61|540.454|airplane_front”,
“0|1244.37|154.76|1402.77|375.957|Jet Bridge Disonnected”
]
}
{
“version” : “4.0”,
“id” : “1150”,
“@timestamp” : “2023-08-21T16:31:07.130Z”,
“sensorId” : “42BE433E-F36B-1410-897A-0039B11BCDE8”,
“description” : “This is camera=42BE433E-F36B-1410-897A-0039B11BCDE8 of Gate=E9BD433E-F36B-1410-897A-0039B11BCDE8 of Terminal=1”,
“gate_id” : “E9BD433E-F36B-1410-897A-0039B11BCDE8”,
“objects” : [
“7|789.78|501.91|1100.86|730.495|Push Back Vehicle Disconnected”,
“4|855.748|165.992|1111.61|540.454|airplane_front”,
“0|1244.37|154.76|1402.77|375.957|Jet Bridge Disonnected”
]
}

can you tell me why it is happening like this?

fanzh · August 22, 2023, 2:11am

msg-conv-msg2p-new-api: 0
msg-conv-frame-interval: 50
msg-conv-frame-interval is invalid if msg-conv-msg2p-new-api is 0, please refer to doc. if msg-conv-msg2p-new-api is 0, every object will be sent. that is not duplicate message, the object id and rectangle are not the same.

psharma3 · August 22, 2023, 4:46am

msg-conv-payload-type: 257
msg-conv-msg2p-new-api: 1
msg-conv-frame-interval: 50
i m using these parameter , in that case also i m getting dulpicates message

code modified mentioned below for custom schema

else if (ctx->payloadType == NVDS_PAYLOAD_CUSTOM){

message = generate_dsmeta_message_minimal(ctx->privData, frame_meta);
if (message) {
  len = strlen (message);
  // Remove '\0' character at the end of string and just copy the content.
  payload->payload = g_memdup (message, len);
  payload->payloadSize = len;
  g_free (message);
}

}

fanzh · August 22, 2023, 9:45am

There is no update from you for a period, assuming this is not an issue any more. Hence we are closing this topic. If need further support, please open a new one. Thanks.
can you print the frame number(frame_num) and source_id in generate_dsmeta_message_minimal? please refer to struct NvDsFrameMeta.
pelase check if the duplicate objects have the sample frame number.

system · September 6, 2023, 12:50am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to change the frequency of payload generation interval in deepstream 6.2? DeepStream SDK nvbugs	15	525	August 13, 2023
Deeepstream-test5 error when save image DeepStream SDK	35	2715	October 12, 2021
New NvStreammux shows 「[ERROR push 317] push failed [-5]」 DeepStream SDK	9	577	June 4, 2024
Custom schema Json output DeepStream SDK	13	431	July 13, 2023
What are the detailed specifications of the properties of new nvstreammux configuration file? DeepStream SDK	7	317	June 25, 2024
Low camera frame rate DeepStream SDK camera , nvbugs	30	6189	October 12, 2021
Error in Deepstream 6.1 DeepStream SDK deepstream	21	502	June 6, 2024
Add Frame to MSGBroker payload using EventMetaData DeepStream SDK nvbugs	37	1374	December 8, 2023
Deepstream Occupancy Analytics Maximum Number of Cameras DeepStream SDK	8	898	December 28, 2021
Slow decoding T4 DS 5.0.1 DeepStream SDK	11	474	October 12, 2021

Deepstream 6.2 , message payload issue

Related topics