Holistically-Nested Edge Detection using TensoRT

[ariel27593]

Please provide complete information as applicable to your setup.

• Hardware Platform Jetson Xavier
• DeepStream Version 6.3.0
• JetPack Version 5.1.2
**• TensorRT Version 8.5.2.2 **

• Issue Type:questions/bugs

Hi,
I try accelerate Deep-edge detection HED - (Holistically-Nested Edge) with TensorRT on my jetson Xavier. I use Pytorch implementation from pytorch-hed extract the model to onnx with constant input=1x3x320x320 and output=1x1x320x320, useing trtexec command line extract engine file. Here is the command: trtexec --onnx=HED.onnx --saveEngine=HED.engine --fp16 --noDataTransfers --useSpinWait.
At inference time I get my results shifted to the bottom right although in Pytorch implementation I don’t get this problem, I know in old Caffe model with Opencv implementation they add CropLayer but I not understand why I get shifted output edge image because I using Pytorch implementation. I would glad to help how I can fix this problem.

[SivaRamaKrishnaNV]

Dear @ariel27593 ,
Does this still need support?
Could you share the ONNX and TRT API code used for test to repro the issue?

[ariel27593]

Hi,
Indeed still relevant attached a zip file with ONNX and relevant inference code.
HED_TRT_PROJ.zip (52.2 MB)

[ariel27593]

@SivaRamaKrishnaNV

Hi, still the problem exists is there any suggestion on how to solve this problem?

[SivaRamaKrishnaNV]

I notice an image like below with your script.

Is the same code working well with caffe → TRT model and noticed issue with PyTorch → ONNX → TRT model?

Also, I see below output with polygraphy tool

[I]         PASSED | Output: 'output' | Difference is within tolerance (rel=0.001, abs=0.001)
[I]     PASSED | All outputs matched | Outputs: ['output']
[I] Accuracy Summary | trt-runner-N0-02/18/25-16:45:54 vs. onnxrt-runner-N0-02/18/25-16:45:54 | Passed: 1/1 iterations | Pass Rate: 100.0%
[I] PASSED | Runtime: 117.463s | Command: /home/nvidia/.local/bin/polygraphy run HED.onnx --trt --onnxrt --atol 0.001 --rtol 0.001 --trt-outputs mark all --no-save-all-tensors-to-one-file --save-external-data /home/nvidia/ --verbose

Looks like some issue with the ONNX model.

[ariel27593]

@SivaRamaKrishnaNV Thank you for replying, I convert to engine file directly from Caffe model Here is trtexec command:

trtexec --model=hed_pretrained_bsds.caffemodel --deploy=deploy.prototxt --output=sigmoid-fuse --saveEngine=HED_Caffe.engine --fp16 --noDataTransfers --useSpinWait --verbose --batch=1 --maxBatch=1

But Now when I try run cpp inference I get the following error although I specify maxBatch=1

Engine loaded from /home/uvision/HED_TRT_PROJECT/aux_folder/HED_Caffe.engine
3: [executionContext.cpp::enqueueV3::2381] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::enqueueV3::2381, condition: !mEngine.hasImplicitBatchDimension(). EnqueueV3 is not supported for network created with implicit batch dimension.
)

and I get black image When I use the onnx file I succussed to output image just with a shift I will be glad to your onion.

In General I not sure the problem is with the onnx file, attaching results with my code and onnx filde (last time),:

Best

[AastaLLL]

Hi,

Just check the source code.
Could you also dump the data after pre-processing and share it with us?

hed.PreProcessImage(img);

Thanks.

[ariel27593]

I found the problem in Preprocessing method, I was need to upload the vector to the gpu in one chunk.