Originally published at: Creating a Real-Time License Plate Detection and Recognition App | NVIDIA Technical Blog
Automatic license plate recognition (ALPR) on stationary to fast-moving vehicles is one of the common intelligent video analytics applications for smart cities. Some of the common use cases include parking assistance systems, automated toll booths, vehicle registration and identification for delivery and logistics at ports, and medical supply transporting warehouses. Being able to do this…
1 Like
The whole process is too complicated. Too many dependencies.
Removed
Could you share the algorithm or implementation of the LPRNet?
I am trying to “Convert the encrypted LPR ONNX model to a TLT engine”, however, in running the command:
./tlt-converter -k nvidia_tlt -p image_input,1x3x48x96,4x3x48x96,16x3x48x96 ./us_lprnet_baseline18_deployable.etltunpruned.etlt -t fp16 -e /opt/nvidia/deepstream/deepstream-5.1/samples/models/LP/LPR/lpr_us_onnx_b16.engine
I am receiving the error:
Error: no input dimensions given
Do you happen to know the dimensions we should be using for this?
cc/ @Morganh (the github repo seems to have the same issues as noted here
cc/ @TomNVIDIA
@bbb
Can you run following command
$ ./tlt-converter -h
Did you download the latest version of tlt-converter ?
See Overview — TAO Toolkit 3.22.05 documentation
or Overview — TAO Toolkit 3.22.05 documentation
Yes, I can run it and it runs as it should. Output:
bryan@bryan-desktop:~/cuda10.2_trt7.1_jp4.5$ ./tlt-converter -h
usage: ./tlt-converter [-h] [-v] [-e ENGINE_FILE_PATH]
[-k ENCODE_KEY] [-c CACHE_FILE]
[-o OUTPUTS] [-d INPUT_DIMENSIONS]
[-b BATCH_SIZE] [-m MAX_BATCH_SIZE]
[-w MAX_WORKSPACE_SIZE] [-t DATA_TYPE]
[-i INPUT_ORDER] [-s] [-u DLA_CORE]
input_file
Generate TensorRT engine from exported model
positional arguments:
input_file Input file (.etlt exported model).
required flag arguments:
-d comma separated list of input dimensions(not required for TLT 3.0 new models).
-k model encoding key.
optional flag arguments:
-b calibration batch size (default 8).
-c calibration cache file (default cal.bin).
-e file the engine is saved to (default saved.engine).
-i input dimension ordering -- nchw, nhwc, nc (default nchw).
-m maximum TensorRT engine batch size (default 16). If meet with out-of-memory issue, please decrease the batch size accordingly.
-o comma separated list of output node names (default none).
-p comma separated list of optimization profile shapes in the format <input_name>,<min_shape>,<opt_shape>,<max_shape>, where each shape has the format: <n>x<c>x<h>x<w>. Can be specified multiple times if there are multiple input tensors for the model. This argument is only useful in dynamic shape case.
-s TensorRT strict_type_constraints flag for INT8 mode(default false).
-t TensorRT data type -- fp32, fp16, int8 (default fp32).
-u Use DLA core N for layers that support DLA(default = -1, which means no DLA core will be utilized for inference. Note that it'll always allow GPU fallback).
-w maximum workspace size of TensorRT engine (default 1<<30). If meet with out-of-memory issue, please increase the workspace size accordingly.
I downloaded the latest version of the tlt-converter as well.
To note, I am starting the tutorial from the bottom section: Deploying LPD and LPR using the DeepStream SDK. Which should be ok to do.
Submitting another comment, in addition to the above one @Morganh
I decided to try another route, in cloning this repo here: GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream
I am able to get the input_dimensions to load correctly. However, it appears another issue is springing up. Multiple commands/outputs here:
Successful/no issue
bryan@bryan-desktop:~/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app$ ../../../../../../../cuda10.2_trt7.1_jp4.5/tlt-converter -k nvidia_tlt -p image_input,1x3x48x96,4x3x48x96,16x3x48x96 \
> models/LP/LPR/us_lprnet_baseline18_deployable.etlt -t fp16 -e models/LP/LPR/lpr_us_onnx_b16.engine
[WARNING] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] Tensor DataType is determined at build time for tensors not marked as input or output.
[INFO] Detected input dimensions from the model: (-1, 3, 48, 96)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 48, 96) for input: image_input
[INFO] Using optimization profile opt shape: (4, 3, 48, 96) for input: image_input
[INFO] Using optimization profile max shape: (16, 3, 48, 96) for input: image_input
[INFO] Detected 1 inputs and 2 output network tensors.
Successful/no issue
bryan@bryan-desktop:~/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app$ make
make[1]: Entering directory '/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/nvinfer_custom_lpr_parser'
g++ -o libnvdsinfer_custom_impl_lpr.so nvinfer_custom_lpr_parser.cpp -Wall -Werror -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -Wl,--start-group -lnvinfer -lnvparsers -Wl,--end-group
make[1]: Leaving directory '/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/nvinfer_custom_lpr_parser'
make[1]: Entering directory '/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app'
g++ -c -o deepstream_lpr_app.o -fpermissive -Wall -Werror -DPLATFORM_TEGRA -I/opt/nvidia/deepstream/deepstream/sources/includes `pkg-config --cflags gstreamer-1.0` -D_GLIBCXX_USE_CXX11_ABI=1 deepstream_lpr_app.c
g++ -c -o deepstream_nvdsanalytics_meta.o -Wall -Werror -DPLATFORM_TEGRA -I/opt/nvidia/deepstream/deepstream/sources/includes `pkg-config --cflags gstreamer-1.0` -D_GLIBCXX_USE_CXX11_ABI=1 deepstream_nvdsanalytics_meta.cpp
cc -o deepstream-lpr-app deepstream_lpr_app.o deepstream_nvdsanalytics_meta.o `pkg-config --libs gstreamer-1.0` -L/opt/nvidia/deepstream/deepstream/lib/ -lnvdsgst_meta -lnvds_meta -lm -lstdc++ -Wl,-rpath,/opt/nvidia/deepstream/deepstream/lib/
make[1]: Leaving directory '/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app'
Unsuccessful/issue
bryan@bryan-desktop:~/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 2 0 us_car_test2.mp4 us_car_test2.mp4 output.264
Request sink_0 pad from streammux
Request sink_1 pad from streammux
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Unknown or legacy key specified 'process_mode' for group [property]
Now playing: 1
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:12.783947731 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 3]: deserialized trt engine from :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
INFO: [FullDims Engine Info]: layers num: 3
0 INPUT kFLOAT image_input 3x48x96 min: 1x3x48x96 opt: 4x3x48x96 Max: 16x3x48x96
1 OUTPUT kINT32 tf_op_layer_ArgMax 24 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT tf_op_layer_Max 24 min: 0 opt: 0 Max: 0
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_bbox/BiasAdd
0:00:12.784171641 8708 0x558f5fb230 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_bbox/BiasAdd' in engine
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_cov/Sigmoid
0:00:12.784218673 8708 0x558f5fb230 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_cov/Sigmoid' in engine
0:00:12.784243465 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 3]: Use deserialized engine model: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
0:00:12.825354703 8708 0x558f5fb230 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine2> [UID 3]: Load new model:lpr_config_sgie_us.txt sucessfully
0:00:12.825646583 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 2]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:01:46.893865326 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 2]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPD/usa_pruned.etlt_b16_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x480x640
1 OUTPUT kFLOAT output_bbox/BiasAdd 4x30x40
2 OUTPUT kFLOAT output_cov/Sigmoid 1x30x40
0:01:47.053010315 8708 0x558f5fb230 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine1> [UID 2]: Load new model:lpd_us_config.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
!! [WARNING][NvDCF] Unknown param found: minMatchingScore4Motion
[NvDCF][Warning] `minTrackingConfidenceDuringInactive` is deprecated
!! [WARNING][NvDCF] Unknown param found: matchingScoreWeight4Motion
[NvDCF] Initialized
ERROR: Deserialize engine failed because file path: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine open error
0:01:49.219258844 8708 0x558f5fb230 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed
0:01:49.219294521 8708 0x558f5fb230 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed, try rebuild
0:01:49.219326814 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:02:36.543306401 8708 0x558f5fb230 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 1]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 16x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 4x34x60
0:02:36.577236606 8708 0x558f5fb230 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-infer-engine1> [UID 1]: Load new model:trafficamnet_config.txt sucessfully
[NvDCF] De-initialized
Running...
ERROR from element file_src_1: Resource not found.
Error details: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstFileSrc:file_src_1:
No such file "us_car_test2.mp4"
Returned, stopping playback
Average fps 0.000233
Totally 0 plates are inferred
Deleting pipeline
You have generated lpr_us_onnx_b16.engine successfully. There is no error in the log.
@Morganh I hit submit too quickly, and needed to edit my above comment. Could you possibly see/review/help me with the unsuccessful point I am seeing above? It’s in building/running the application per github commands here:
I move this topic from content discussions–developer blog to TLT forum.
1 Like
Please check if us_car_test2.mp4 is available.
I should have posted the log below as I noticed that “us_car_test2.mp4” might not be available. Video I am trying to test with is within the deepstream-5.1/samples/streams (sample_qHD.mp4). It appears the logs are similar:
bryan@bryan-desktop:~/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 2 0 ../../../streams/sample_qHD.mp4 ../../../streams/sample_qHd.mp4 output.264
Request sink_0 pad from streammux
Request sink_1 pad from streammux
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Unknown or legacy key specified 'process_mode' for group [property]
Now playing: 1
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:11.534274321 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 3]: deserialized trt engine from :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
INFO: [FullDims Engine Info]: layers num: 3
0 INPUT kFLOAT image_input 3x48x96 min: 1x3x48x96 opt: 4x3x48x96 Max: 16x3x48x96
1 OUTPUT kINT32 tf_op_layer_ArgMax 24 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT tf_op_layer_Max 24 min: 0 opt: 0 Max: 0
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_bbox/BiasAdd
0:00:11.534479118 18445 0x5592fe2830 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_bbox/BiasAdd' in engine
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_cov/Sigmoid
0:00:11.534532088 18445 0x5592fe2830 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_cov/Sigmoid' in engine
0:00:11.534562922 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 3]: Use deserialized engine model: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
0:00:11.576047481 18445 0x5592fe2830 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine2> [UID 3]: Load new model:lpr_config_sgie_us.txt sucessfully
0:00:11.576306498 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 2]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:01:46.495432090 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 2]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/LP/LPD/usa_pruned.etlt_b16_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x480x640
1 OUTPUT kFLOAT output_bbox/BiasAdd 4x30x40
2 OUTPUT kFLOAT output_cov/Sigmoid 1x30x40
0:01:46.743905225 18445 0x5592fe2830 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine1> [UID 2]: Load new model:lpd_us_config.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
!! [WARNING][NvDCF] Unknown param found: minMatchingScore4Motion
[NvDCF][Warning] `minTrackingConfidenceDuringInactive` is deprecated
!! [WARNING][NvDCF] Unknown param found: matchingScoreWeight4Motion
[NvDCF] Initialized
ERROR: Deserialize engine failed because file path: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine open error
0:01:48.909930484 18445 0x5592fe2830 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed
0:01:48.909990381 18445 0x5592fe2830 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed, try rebuild
0:01:48.910035643 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:02:38.473025813 18445 0x5592fe2830 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 1]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 16x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 4x34x60
0:02:38.647800783 18445 0x5592fe2830 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-infer-engine1> [UID 1]: Load new model:trafficamnet_config.txt sucessfully
[NvDCF] De-initialized
Running...
ERROR from element file_src_1: Resource not found.
Error details: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstFileSrc:file_src_1:
No such file "../../../streams/sample_qHd.mp4"
Returned, stopping playback
Average fps 0.000233
Totally 0 plates are inferred
Deleting pipeline
Please check the mp4 file is available.
I am seeing this file in my terminal (it came with deepstream). I used autocomplete (tab) to get the file.
@Morganh Something kinda interesting. I created another clone to re-run the tlt converter to add an output.
Previous Command
./tlt-converter -k nvidia_tlt -p image_input,1x3x48x96,4x3x48x96,16x3x48x96 models/LP/LPR/us_lprnet_baseline18_deployable.etlt -t fp16 -e models/LP/LPR/lpr_us_onnx_b16.engine
New Command: adding -o output_cov/Sigmoid,output_bbox/BiasAdd
./tlt-converter -k nvidia_tlt -o output_cov/Sigmoid,output_bbox/BiasAdd -p image_input,1x3x48x96,4x3x48x96,16x3x48x96 models/LP/LPR/us_lprnet_baseline18_deployable.etlt -t fp16 -e models/LP/LPR/lpr_us_onnx_b16.engine
New unsuccessful command to build/run, but maybe right direction?
bryan@bryan-desktop:~/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/deepstream-lpr-app$ ./deepstream-lpr-app 1 2 0 ../../../../streams/sample_720p.mp4 ../../../../streams/sample_720p.mp4 output.264
Request sink_0 pad from streammux
Request sink_1 pad from streammux
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Unknown or legacy key specified 'process_mode' for group [property]
Now playing: 1
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:10.921601849 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 3]: deserialized trt engine from :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
INFO: [FullDims Engine Info]: layers num: 3
0 INPUT kFLOAT image_input 3x48x96 min: 1x3x48x96 opt: 4x3x48x96 Max: 16x3x48x96
1 OUTPUT kINT32 tf_op_layer_ArgMax 24 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT tf_op_layer_Max 24 min: 0 opt: 0 Max: 0
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_bbox/BiasAdd
0:00:10.921851646 19412 0x55639b70d0 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_bbox/BiasAdd' in engine
ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_cov/Sigmoid
0:00:10.921900970 19412 0x55639b70d0 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1670> [UID = 3]: Could not find output layer 'output_cov/Sigmoid' in engine
0:00:10.921961909 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 3]: Use deserialized engine model: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/models/LP/LPR/lpr_us_onnx_b16.engine
0:00:10.962857824 19412 0x55639b70d0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine2> [UID 3]: Load new model:lpr_config_sgie_us.txt sucessfully
0:00:10.963561485 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 2]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:01:43.118323033 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 2]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/models/LP/LPD/usa_pruned.etlt_b16_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x480x640
1 OUTPUT kFLOAT output_bbox/BiasAdd 4x30x40
2 OUTPUT kFLOAT output_cov/Sigmoid 1x30x40
0:01:43.281237925 19412 0x55639b70d0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-infer-engine1> [UID 2]: Load new model:lpd_us_config.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
!! [WARNING][NvDCF] Unknown param found: minMatchingScore4Motion
[NvDCF][Warning] `minTrackingConfidenceDuringInactive` is deprecated
!! [WARNING][NvDCF] Unknown param found: matchingScoreWeight4Motion
[NvDCF] Initialized
ERROR: Deserialize engine failed because file path: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine open error
0:01:45.437559981 19412 0x55639b70d0 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed
0:01:45.437606544 19412 0x55639b70d0 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/deepstream-lpr-app/../models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine failed, try rebuild
0:01:45.437640139 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
WARNING: INT8 not supported by platform. Trying FP16 mode.
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
0:02:33.741014630 19412 0x55639b70d0 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1749> [UID = 1]: serialize cuda engine to file: /home/bryan/opt/nvidia/deepstream/deepstream-5.1/samples/models/test/deepstream_lpr_app/models/tlt_pretrained_models/trafficcamnet/resnet18_trafficcamnet_pruned.etlt_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 16x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 4x34x60
0:02:34.035270946 19412 0x55639b70d0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-infer-engine1> [UID 1]: Load new model:trafficamnet_config.txt sucessfully
Running...
qtdemux pad video/x-h264
qtdemux pad video/x-h264
h264parser already linked. Ignoring.
h264parser already linked. Ignoring.
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
Frame Number = 0 Vehicle Count = 0 Person Count = 0 License Plate Count = 0
Frame Number = 1 Vehicle Count = 0 Person Count = 0 License Plate Count = 0
Frame Number = 2 Vehicle Count = 0 Person Count = 0 License Plate Count = 0
Frame Number = 3 Vehicle Count = 6 Person Count = 4 License Plate Count = 0
Frame Number = 4 Vehicle Count = 6 Person Count = 4 License Plate Count = 0
Frame Number = 5 Vehicle Count = 6 Person Count = 4 License Plate Count = 0
Frame Number = 6 Vehicle Count = 6 Person Count = 4 License Plate Count = 0
Frame Number = 7 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 8 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 9 Vehicle Count = 10 Person Count = 4 License Plate Count = 0
Frame Number = 10 Vehicle Count = 10 Person Count = 4 License Plate Count = 0
Frame Number = 11 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 12 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 13 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 14 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 15 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 16 Vehicle Count = 8 Person Count = 4 License Plate Count = 0
Frame Number = 17 Vehicle Count = 10 Person Count = 4 License Plate Count = 0
open dictionary file failed.
0:02:40.895026375 19412 0x5563261d40 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Error in NvDsInferContextImpl::fillClassificationOutput() <nvdsinfer_context_impl_output_parsing.cpp:796> [UID = 3]: Failed to parse classification attributes using custom parse function
open dictionary file failed.
0:02:40.895353049 19412 0x5563261d40 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<secondary-infer-engine2> NvDsInferContext[UID 3]: Error in NvDsInferContextImpl::fillClassificationOutput() <nvdsinfer_context_impl_output_parsing.cpp:796> [UID = 3]: Failed to parse classification attributes using custom parse function
Segmentation fault (core dumped)
Alike that in the previous log/error, I see this still: ERROR: [TRT]: INVALID_ARGUMENT: Cannot find binding of given name: output_bbox/BiasAdd
Please ignore " Cannot find binding of given name: output_bbox/BiasAdd"
For the latest error “open dictionary file failed”, please follow
For US car plate recognition
cp dict_us.txt dict.txt
For Chinese car plate recognition
cp dict_ch.txt dict.txt
It seems as though it is running correctly, now (on my revised tlt-converter command; which is still a bit odd, but it’s late here and I might be getting a bit fatigued).
Last question (I hope), which might be a bit dumb: the output of the results is only in terminal, is it possible to view the mp4 video as well?
See
./deepstream-lpr-app <1:US car plate model|2: Chinese car plate model> \
<1: output as h264 file| 2:fakesink 3:display output> <0:ROI disable|1:ROI enable> \
<input mp4 file name> ... <input mp4 file name> <output file name>
Please run with ./deepstream-lpr-app 1 1 0 …
@Morganh Thanks for all the help. Got it working with 1 3 0