Failed to create .engine File

anshul12256 · June 3, 2022, 2:24pm

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) : Jetson Xavier NX
• DeepStream Version :6 .0
• JetPack Version (valid for Jetson only) : Jetpack 4.6 (32.6.1)
• TensorRT Version : 8.0.6.1
• Issue Type( questions, new requirements, bugs) - Question

Hello Everyone,
I have a deepstream application with 4 input video file sources. In the pgie_config.txt file, I am using an efficientdet0 model. When I set the batch size to 1, the engine file is created successfully. However, when I put the batch size to 4 it fails to create the engine file and gives the following error:

ERROR: [TRT]: 4: [shapeCompiler.cpp::evaluateShapeChecks::822] 
Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: reshape would change volume. IShuffleLayer block1a_se_squeeze/Mean_Squeeze__3549: reshaping failed for tensor: block1a_se_squeeze/Mean:0)
ERROR: Build engine failed from config file
Segmentation fault

Following is my pgie_config.txt file:

[property]
labelfile-path= ../../models/people_net_model/labels.txt
enable-dla=0
use-dla-core=0
infer-dims=3;544;960
gpu-id=0
gie-unique-id=1
model-color-format=0
network-mode=1      ## 0=FP32, 1=INT8, 2=FP16 mode
maintain-aspect-ratio=1
#batch-size=1
network-input-order=1
net-scale-factor=1.0
offsets=0;0;0
is-classifier=0
cluster-mode=4      

int8-calib-file=../../models/people_net_model/efficientdet_d0.cal
tlt-encoded-model = ../../models/people_net_model/det0_int8.etlt

tlt-model-key=nvidia_tlt
uff-input-blob-name=image_arrays:0
output-blob-names=num_detections;detection_boxes;detection_scores;detection_classes
parse-bbox-func-name=NvDsInferParseCustomEfficientDetTAO
custom-lib-path = /home/nvidia/deepstream_tao_apps/post_processor/libnvds_infercustomparser_tao.so
num-detected-classes=1
interval=2

[class-attrs-all]
pre-cluster-threshold=0.15
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

Can you tell me how to create the engine file for 4 or more sources?

Looking forward to your replies

Fiona.Chen · June 6, 2022, 8:13am

The pretrained model’s max batch size is 1. Please re-train the model with bigger batch size. EfficientDet — TAO Toolkit 3.22.05 documentation (nvidia.com)

anshul12256 · June 6, 2022, 4:50pm

Hello @Fiona.Chen,
Thank you for your reply. Can you tell me how to specify this max batch size while training?
I was not able to find the option.

I am looking forward to your reply.

Fiona.Chen · June 7, 2022, 2:15am

@Morganh Can you help @anshul12256 to figure out how to modify the batch size of efficientdet0?

Morganh · June 7, 2022, 2:28am

@anshul12256
User can set train_batch_size during training. See EfficientDet — TAO Toolkit 3.22.05 documentation

But it does not make sense when you “put the batch size to 4 it fails to create the engine file and gives the following error” . Please share the full command and full log.

Morganh · June 7, 2022, 2:42am

Moving to TAO forum.

anshul12256 · June 7, 2022, 7:09am

Hello @Morganh,
The train_batch_size parameter is used to set the batch size of the training dataset for training the model.
I want to create an engine file from the .etlt model file for 4 input sources.
These are two different things.
In my pgie_config file, I set the path to the etlt model and when I launch my deepstream application I get the following messages on the terminal

nvidia@nvidia-desktop:~/embeddedvision$ ./embedded_vision
PIPELINE 1 = streammux -> pgie -> nvtracker -> tiler -> nvosd
nvosd -> tee
tee -> output_sink_bin
greenMenu
Get Video Handle
Starting...

Using winsys: x11
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[/dvs/git/dirty/git-master_linux/deepstream/sdk/src/utils/nvmultiobjecttracker/include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 54]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
~~ CLOG[/dvs/git/dirty/git-master_linux/deepstream/sdk/src/utils/nvmultiobjecttracker/include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 54]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
[NvMultiObjectTracker] Initialized
0:00:00.756406906 29730   0x5597d72c90 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 97) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 208) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 211) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 309) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 312) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 316) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 319) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 326) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 333) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 339) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 351) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 356) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 362) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 374) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 379) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 385) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 397) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 402) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 408) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 420) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 423) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 430) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 442) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 445) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 448) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 456) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 468) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 471) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 474) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 482) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 494) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 497) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 500) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 508) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 520) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 525) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 531) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 543) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 548) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 554) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 566) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 571) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 577) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 589) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 594) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 600) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 612) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 615) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 622) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 634) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 637) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 640) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 648) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 660) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 663) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 666) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 674) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 686) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 689) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 692) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 700) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 712) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 717) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 723) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 735) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 740) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 746) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 758) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 763) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 769) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 781) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 786) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 792) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 807) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 821) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 854) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 876) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 909) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 931) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 964) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 986) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead
ERROR: [TRT]: 4: [shapeCompiler.cpp::evaluateShapeChecks::822] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: reshape would change volume. IShuffleLayer block1a_se_squeeze/Mean_Squeeze__3549: reshaping failed for tensor: block1a_se_squeeze/Mean:0)
ERROR: Build engine failed from config file
Segmentation fault

Morganh · June 7, 2022, 7:42am

What is the “embedded_vision”?

anshul12256 · June 7, 2022, 7:50am

Hello @Morganh,
That is the name of my deepstream application.

Morganh · June 7, 2022, 8:19am

Could you please try default official deepstream application?

anshul12256 · June 8, 2022, 3:46pm

Hello @Morganh, I tried the default official tao application but even that worked with only a batch size of 1. However, I solved the problem by using the following command in the efficientdet jupyter notebook

!mkdir -p $LOCAL_EXPERIMENT_DIR/export_int8
# Uncomment to remove existing etlt file
# !rm $LOCAL_EXPERIMENT_DIR/experiment_dir_retrain/model.step-0.etlt
!tao efficientdet export -m $USER_EXPERIMENT_DIR/experiment_dir_retrain/model.step-$NUM_STEP.tlt \
                         -o $USER_EXPERIMENT_DIR/experiment_dir_retrain/model_int8.step-$NUM_STEP.etlt \
                         -k $KEY \
                         -e $SPECS_DIR/efficientdet_d1_retrain.txt \
                         --batch_size 4 \
                         --data_type int8 \
                         --cal_image_dir $DATA_DOWNLOAD_DIR/training_v6_coco_crowd/images_testing \
                         --batches 10 \
                         --max_batch_size 4  \
                         --cal_cache_file $USER_EXPERIMENT_DIR/export_int8/efficientdet_d0.cal

Here the max_batch_size parameter enabled to create the .etlt model with a batch size of 4.

However, now I have another question.
When I provided the above .etlt model with a batch size of 4 in the pgie_config.txt file, the resulting engine file created by the system is of 27.2 MB. My .etlt model is 9.3 MB. Since the engine file is huge, I get very poor performance (6-15 FPS) for 4 streams.

To verify this, I also used the tao-converter on the XNX with the following command:

./tao-converter -k nvidia_tlt -d 544,960,3 -c /home/nvidia/embeddedvision/models/people_net_model/efficientdet_d2.cal -m 4 -t int8 /home/nvidia/embeddedvision/models/people_net_model/model_int8.step-562500.etlt

The resulting engine file created is also around the same size of 26.2 MB.

I would like to know why the int8 engine is nearly 3X bigger than the model?

Because when I used detectnetv2 with resnet34 as the backbone architecture the etlt model was 10.4 MB and the resulting int8 engine file was 5.8 MB

Looking forward to your response

anshul12256 · June 13, 2022, 8:22am

Hello @Morganh,
Would like to ask, were you able to get the solution to the above question?

Morganh · June 13, 2022, 10:08am

How about the fp16 engine’s size and its fps?
More, usually, the fps is related to trainable parameters.

anshul12256 · June 13, 2022, 1:12pm

Hello @Morganh,
For the FP16 the engine size was 19.3 MB with 17-20 FPS for 1 stream. I couldn’t check for 4 streams because I had exported the model with a max batch size of 1.

I find this a bit surprising that an FP16 engine is smaller than an INT8 engine!

Would like to know your opinion !!

Morganh · June 13, 2022, 1:27pm

I need to reproduce firstly.

anshul12256 · June 13, 2022, 1:31pm

Hi @Morganh,
Thanks for your reply. Looking forward to your findings !!
I am using efficientdet-d2 as the model name and b2 as the backbone architecture.

Morganh · June 15, 2022, 9:20am

How did you run against 4 streams? Can you share the steps?

anshul12256 · June 15, 2022, 9:43am

Hello @Morganh,
For int8 i had exported the etlt model with a batch size of 4 using the command in the earlier post.
For fp16 there should be a parameter I suppose which would help to export the etlt model with a batch size of 4

For running on 4 Streams, I have a custom deepstream application. Its pipeline is as follows

filesrc(source=0, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=1, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=2, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=3, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
nvstreammux -> nvinfer -> nvmultistreamtiler -> nvdsosd -> nvegltransform -> nveglglessink

Morganh · June 15, 2022, 10:38am

I cannot reproduce the result official .etlt file and cal file.

$ ./tao-converter -k nvidia_tlt -d 544,960,3 d0_avlp_544_960.etlt -t fp16 -e fp16.engine

$ ./tao-converter -k nvidia_tlt -d 544,960,3 d0_avlp_544_960.etlt -t int8 -c d0_avlp_544_960.cal -e int8.engine

Or login 22.05 tf15.5 docker,

# converter -k nvidia_tlt -d 544,960,3 d0_avlp_544_960.etlt -t int8 -c d0_avlp_544_960.cal -e int8.engine

# converter -k nvidia_tlt -d 544,960,3 d0_avlp_544_960.etlt -t fp16 -e fp16.engine

Actually you can focus on the pruning ratio. For better fps, please prune to a smaller one and retrain it.

anshul12256 · June 15, 2022, 10:44am

Hello @Morganh,
what are the sizes of the engine file that you created from the official etlt model ?

Topic		Replies	Views
EfficientDet in Deepstream Causes a Seg Fault TAO Toolkit efficientdet , tao	15	1058	July 19, 2022
Deepstream-gaze-app segmentation fault DeepStream SDK	10	255	February 13, 2024
Bpnet sample code error TAO Toolkit	13	768	October 11, 2022
DeepStream 6.4 Python App ERROR DeepStream SDK deepstream	3	453	December 25, 2023
Tao-converter doesn't work for Deepstream 6.1 TAO Toolkit	7	754	July 14, 2022
Deepstream infrence gives no detection TAO Toolkit	28	1927	December 9, 2021
Fine-tuned TAO ClassificationTF2 Accuracy Drop after Compiling to TensorRT TAO Toolkit	34	764	August 6, 2024
Is the deepstream_lpr_app supposed to work with DS 7.0? DeepStream SDK	7	224	May 17, 2024
TAO toolkit fails to convert RetinaNet INT8 etlt model to INT8 CUDA engine (calibration cache needs to be deleted?) TAO Toolkit tensorrt , cuda	4	450	June 10, 2022
TAO 21.11 detectnet_v2 fallback to fp16 in DS6 TAO Toolkit tao , deepstream	8	2164	January 25, 2022

Failed to create .engine File

Related topics