Change class number of yolov3 in DeepStream

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Xavier
• DeepStream Version 4.0.2
• JetPack Version (valid for Jetson only) 4.3
• TensorRT Version 6.0.1
• NVIDIA GPU Driver Version (valid for GPU only)

When I try objectDetector_Yolo project, I change the network model with yolov3 network trained with 4 classes.
I changed the code as follow:
config_infer_primary_yoloV3.txt

num-detected-classes=80 =>>  num-detected-classes=4

label.txt

car
person
bicycle
bus

nvdsparsebbox_Yolo.cpp

static const int NUM_CLASSES_YOLO = 80; =>> static const int NUM_CLASSES_YOLO = 4;

However, when running deepstream-app -c deepstream_app_config_yoloV3.txt, I cna load the model and build the TensorRT Engine, but still have an error of class mismatch. Can you help me?

Using winsys: x11 
Creating LL OSD context new
0:00:00.891588919 12995   0x7f1c002240 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 61592497
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 416 x 416      32 x 416 x 416    992   
(2)   conv-bn-leaky    32 x 416 x 416      64 x 208 x 208    19680 
(3)   conv-bn-leaky    64 x 208 x 208      32 x 208 x 208    21856 
(4)   conv-bn-leaky    32 x 208 x 208      64 x 208 x 208    40544 
(5)   skip             64 x 208 x 208      64 x 208 x 208        - 
(6)   conv-bn-leaky    64 x 208 x 208     128 x 104 x 104    114784
(7)   conv-bn-leaky   128 x 104 x 104      64 x 104 x 104    123232
(8)   conv-bn-leaky    64 x 104 x 104     128 x 104 x 104    197472
(9)   skip            128 x 104 x 104     128 x 104 x 104        - 
(10)  conv-bn-leaky   128 x 104 x 104      64 x 104 x 104    205920
(11)  conv-bn-leaky    64 x 104 x 104     128 x 104 x 104    280160
(12)  skip            128 x 104 x 104     128 x 104 x 104        - 
(13)  conv-bn-leaky   128 x 104 x 104     256 x  52 x  52    576096
(14)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    609376
(15)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    905312
(16)  skip            256 x  52 x  52     256 x  52 x  52        - 
(17)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    938592
(18)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1234528
(19)  skip            256 x  52 x  52     256 x  52 x  52        - 
(20)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1267808
(21)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1563744
(22)  skip            256 x  52 x  52     256 x  52 x  52        - 
(23)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1597024
(24)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    1892960
(25)  skip            256 x  52 x  52     256 x  52 x  52        - 
(26)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    1926240
(27)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2222176
(28)  skip            256 x  52 x  52     256 x  52 x  52        - 
(29)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2255456
(30)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2551392
(31)  skip            256 x  52 x  52     256 x  52 x  52        - 
(32)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2584672
(33)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    2880608
(34)  skip            256 x  52 x  52     256 x  52 x  52        - 
(35)  conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    2913888
(36)  conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    3209824
(37)  skip            256 x  52 x  52     256 x  52 x  52        - 
(38)  conv-bn-leaky   256 x  52 x  52     512 x  26 x  26    4391520
(39)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    4523616
(40)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    5705312
(41)  skip            512 x  26 x  26     512 x  26 x  26        - 
(42)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    5837408
(43)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    7019104
(44)  skip            512 x  26 x  26     512 x  26 x  26        - 
(45)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    7151200
(46)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    8332896
(47)  skip            512 x  26 x  26     512 x  26 x  26        - 
(48)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    8464992
(49)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    9646688
(50)  skip            512 x  26 x  26     512 x  26 x  26        - 
(51)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    9778784
(52)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    10960480
(53)  skip            512 x  26 x  26     512 x  26 x  26        - 
(54)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    11092576
(55)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    12274272
(56)  skip            512 x  26 x  26     512 x  26 x  26        - 
(57)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    12406368
(58)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    13588064
(59)  skip            512 x  26 x  26     512 x  26 x  26        - 
(60)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    13720160
(61)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    14901856
(62)  skip            512 x  26 x  26     512 x  26 x  26        - 
(63)  conv-bn-leaky   512 x  26 x  26    1024 x  13 x  13    19624544
(64)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    20150880
(65)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    24873568
(66)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(67)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    25399904
(68)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    30122592
(69)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(70)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    30648928
(71)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    35371616
(72)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(73)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    35897952
(74)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    40620640
(75)  skip           1024 x  13 x  13    1024 x  13 x  13        - 
(76)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    41146976
(77)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    45869664
(78)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    46396000
(79)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    51118688
(80)  conv-bn-leaky  1024 x  13 x  13     512 x  13 x  13    51645024
(81)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    56367712
(82)  conv-linear    1024 x  13 x  13      27 x  13 x  13    56395387
(83)  yolo             27 x  13 x  13      27 x  13 x  13    56395387
(84)  route                  -            512 x  13 x  13    56395387
(85)  conv-bn-leaky   512 x  13 x  13     256 x  13 x  13    56527483
(86)  upsample        256 x  13 x  13     256 x  26 x  26        - 
(87)  route                  -            768 x  26 x  26    56527483
(88)  conv-bn-leaky   768 x  26 x  26     256 x  26 x  26    56725115
(89)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    57906811
(90)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    58038907
(91)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    59220603
(92)  conv-bn-leaky   512 x  26 x  26     256 x  26 x  26    59352699
(93)  conv-bn-leaky   256 x  26 x  26     512 x  26 x  26    60534395
(94)  conv-linear     512 x  26 x  26      27 x  26 x  26    60548246
(95)  yolo             27 x  26 x  26      27 x  26 x  26    60548246
(96)  route                  -            256 x  26 x  26    60548246
(97)  conv-bn-leaky   256 x  26 x  26     128 x  26 x  26    60581526
(98)  upsample        128 x  26 x  26     128 x  52 x  52        - 
(99)  route                  -            384 x  52 x  52    60581526
(100) conv-bn-leaky   384 x  52 x  52     128 x  52 x  52    60631190
(101) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    60927126
(102) conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    60960406
(103) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    61256342
(104) conv-bn-leaky   256 x  52 x  52     128 x  52 x  52    61289622
(105) conv-bn-leaky   128 x  52 x  52     256 x  52 x  52    61585558
(106) conv-linear     256 x  52 x  52      27 x  52 x  52    61592497
(107) yolo             27 x  52 x  52      27 x  52 x  52    61592497
Output blob names :
yolo_83
yolo_95
yolo_107
Total number of layers: 257
Total number of layers on DLA: 0
Building the TensorRT Engine...
Building complete!
0:05:35.819364291 12995   0x7f1c002240 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/nvidia/deepstream_sdk_v4.0.2_jetson/sources/objectDetector_Yolo/model_b1_int8.engine
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF: FPS 0 (Avg)	
**PERF: 0.00 (0.00)	
** INFO: <bus_callback:189>: Pipeline ready

Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
WARNING: Num classes mismatch. Configured:4, detected by network: 80
Segmentation fault (core dumped)

Can you double check your source file if you have infact updated the NUM_CLASSES_YOLO variable ? You will need to rebuild the lib after making the change.

If the issue persists once you verify the change above, can you obtain a backtrace for the seg fault ?

I rebuild the lib and solve the problem, thank you!

Now I have a problem about using deepstream accelerate models in a C++ project. Using tensorRT, I can accelerate models in a project very easily, but I don’t know how can I do that with deepstream. Can you help me?

Hi

Please open a new topic for this new issue. Thanks

Ok, thank you for your reply!