Deepstream SDK - Autoencoder/Detector feature maps

Hello there. I have a design need for an Autoencoder that is trained on feature maps of the detectors. I was wondering how I might go about this. I can can perform the ROI pooling and inference outside of the pipeline, but I need access to the feature maps of whatever detector we end up using in our pipeline. Our current implementation involves sending out the per camera image as a byte array and passing that into a detector outside the deepstream pipeline. Running another detector is clearly not an efficient solution. Any direction would be appreciated, as I cannot find anything online related to this.