How to run tao visual changenet segmentation MVTec model on deepstream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

TensorRT Version: 8.5.3.1
GPU Type: RTX 4060 laptop
Nvidia Driver Version: 530.41.03
CUDA Version: 12.1 Update 1
Operating System + Version: Ubuntu 20.04

Hello NVIDIA community,

I recently trained a Visual ChangeNet segmentation MVTec model for anomaly detection using the TAO Toolkit. After training, I tested the model’s functionality using the tao inference command and successfully generated the .onnx file using the tao export command.

Now, I’m eager to deploy this trained model for anomaly detection in DeepStream. However, I’m facing difficulties in finding reference materials on how to set up the primary config file.

Could someone please guide me on how to create the primary config file to run this model in DeepStream?

1 Like
4

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks! the Visual ChangeNet Segmentation - MvTEC model requires two images as inputs. you need to use nvdspreprocess plugin to package the preprocessed images to tensor, then pass the tensor to nvinfer/nvinferserver.

5

I haven’t been able to solve the issue yet. These are the files I’m using to run the application:

config_infer_primary_changenet.yml.txt (2.6 KB)
deepstream_app_source1_changenet.yml.txt (3.7 KB)

The output I’m getting looks like this:

log.txt (3.3 KB)

I still can’t figure out how to provide a second input using the nvdspreprocess plugin. Could you please provide more details?

6

first you can use gst-launch to debug. the two input pictures need to one-to-one conrrespondence. you can use one preprocess to add one picture’s tensor, then use another preprocess to add the other picture’s tensor. please refer to sample deepstream-preprocess-test for the usage of nvdspreprocess.

gst-launch-1.0 -e multifilesrc location=samples/multifilesrc/"test_%d.jpg" ! jpegparse ! nvjpegdec ! nvvideoconvert interpolation-method=1 ! "video/x-raw(memory:NVMM)" ! mux_0.sink_0 multifilesrc location=samples/multifilesrc/"golden_%d.jpg" ! jpegparse ! nvjpegdec ! nvvideoconvert interpolation-method=1 ! "video/x-raw(memory:NVMM)" ! mux_0.sink_1 nvstreammux name=mux_0 batch-size=2 width=256 height=256 batched-push-timeout=3000000 nvbuf-memory-type=0 live-source=0 ! queue ! nvdspreprocess config-file=configs/config_preprocess_0.txt ! nvdspreprocess config-file=configs/config_preprocess_1.txt ! nvinferserver batch-size=1 config-file-path=configs/config_nvinferserver.txt ! nvsegvisual batch-size=1 height=256 width=256 ! nvmultistreamtiler rows=1 columns=2 width=512 height=256 ! nvvideoconvert ! videorate ! "video/x-raw,framerate=1/2" ! nveglglessink window-x=522
7

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.