Different results from Triton Inference Server and gst-nvinferserver on the same model

AI & Data Science Computer Vision & Image Processing
,
1

I am trying to use the VechicleTypeNet Pruned model to classify the car provided in the image.

The same model was hosted on a triton inference server and was used in a deepstream pipeline with nvinferserver plugin.

config.pbtxt for the model :

platform: "tensorrt_plan"
name : "Vehicle"
max_batch_size: 1
  input [
    {
      name: "input_1"
      data_type: TYPE_FP32
      format:FORMAT_NCHW
      dims: [3,224,224]
    }
  ]
  output [
    {
      name: "predictions/Softmax"
      data_type: TYPE_FP32
      dims: [6,1,1]
    }
  ]

The post processing steps followed before inference using the triton infer server :

img = cv2.resize(img,(224,224))
img4 = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
img4 = img4.astype(np.float32)
img4[:,:,0] -= offset[0]
img4[:,:,1] -= offset[1]
img4[:,:,2] -= offset[2]
img4 = img4*nsf
img4 = np.transpose(img4,(2,0,1))
img4 = np.expand_dims(img4,axis = 0)

offset = [103.939, 116.779, 123.68] and nsf = 1, according the NGC Catalog

The config file attached to the nvinferserver plugin :

infer_config {
  unique_id: 5
  gpu_ids: [0]
  max_batch_size: 1
  backend {
    trt_is {
      model_name: "Vehicle"
      version: -1
      model_repo {
        root: "./"
        log_level: 0
      }
    }
  }

  preprocess {
    network_format: IMAGE_FORMAT_RGB
    tensor_order: TENSOR_ORDER_LINEAR
    maintain_aspect_ratio: 0
    normalize {
      scale_factor: 1
      channel_offsets: [103.939, 116.779, 123.68]
    }
  }

  postprocess {
    labelfile_path: "./Vehicle/labels.txt"
    other {}
  }

  extra {
    copy_input_to_host_buffers: false
  }

  custom_lib {
    path: "/opt/nvidia/deepstream/deepstream/lib/libnvds_infercustomparser.so"
  }
}
input_control {
  process_mode: PROCESS_MODE_FULL_FRAME
  interval: 0
}
output_control {
  output_tensor_meta: true
}

Note: I am using this a primary detector and the input I am giving is only image of a vehicle and it is not part of a Detection Pipeline

Code to get the inference result from the gst-buffer :

    detection_layer = layer_finder(output_layer_info, "predictions/Softmax")

    if not detection_layer :
        sys.stderr.write("ERROR: some layers missing in output tensors\n")
        return []

    if detection_layer.buffer:
        ptr = ctypes.cast(pyds.get_ptr(detection_layer.buffer), ctypes.POINTER(ctypes.c_float))
        res = np.ctypeslib.as_array(ptr,shape = (6,1,1))


    labels = [ "coupe","largevehicle","sedan","suv","truck","van"]
    print('\n\n',"Result Array",res.flatten(),'\n\n')
    print('\n\n','Result Index : ',np.argmax(res.flatten()), '\n\n')
    print('\n\n','Result Name : ',labels[np.argmax(res.flatten())], '\n\n')

The above code is a modified version of the ssd_parser.py from the deepstream-ssd-parser example.

Can someone please help me identify the issue and reason why I am getting different values after inference.