Running Secondary Gie in objectDetector_SSD Deepstream

I am trying to run a primary + secondary gie which are both ssd models. I have successfully converted both the models as stated in the readme file. Both the models run individually as primary gie. But when I add one of the models to secondary gie, it does not deserialise the second model. It only deserialises the model in the primary gie and proceeds to show detection with the primary model.

Following are my configurations in the deepstream_app_config_ssd.txt

[primary-gie]
enable=1
gpu-id=0
batch-size=1
gie-unique-id=1
interval=0
#labelfile-path=ssd_hand_labels.txt
#model-engine-file=ssd_mobilenet_v2_egohands.uff_b1_fp32.engine
config-file=config_infer_primary_ssd.txt
nvbuf-memory-type=0

[secondary-gie0]
enable=1
gpu-id=0
batch-size=1
gie-unique-id=2
interval=0
#labelfile-path=mask_face_labels.txt
#model-engine-file=ssd_mask_face3.uff_b1_fp32.engine
config-file=config_infer_secondary_ssd.txt
nvbuf-memory-type=0

I tried back-to-back detector example as well. I replaced the primary and secondary config files with the ssd primary and secondary config files. I just need to run it on usbcam. Which I am still trying to figure out how to change the source code to achieve that.

So to summarise, secondary gie model is not deserialising/creating at all when I used ssd example.