I’m trying to reconstruct YOLOv2 in TensorRT using the API (no parser).
I’m using the provided YOLORegion layer created by NVIDIA as a plugin layer. The API documentation is however not very clear about its functionality. I notived it also outputs a tensor with the same dimensions as its input tensor, but what do the values mean? Does is only do the sigmoid activations on the coords and softmax operation or does it also calculates the final (relative) box coordinates? Some mathematical explanation would be nice so I know what to do to become a set of boundingboxes that can go to the NMS stage.
I also currently am able to run the whole network except for the region layer. If I exclude the region layer, the network infers well. If I include the region layer, running the network (enqueue call) aborts with:
Cuda failure: 11 at line 116
I initialize the layer with the following parameters when building the engine:
plugin::RegionParameters region_params; region_params.num = 5; region_params.coords = 4; region_params.classes = 80; region_params.smTree = nullptr;
smTree is a nullptr because I’m not using yolo9000, so simple softmax will do.
I’m using TensorRT 126.96.36.199, cudnn 7.1.4 and cuda 8.0.