With regard to using tensorRT with yolov4(tensorflow implement)

Hello,

I have used following github project.
url : https://github.com/hunglc007/tensorflow-yolov4-tflite

here is “convert to tensorRT”.

Can anyone tell me how to use this model with tensorRT?

Thank you.

Hi,

You can run it similar to the sample shared below:

# deserialize engine
with open(myfile, 'rb') as f:
    buf = f.read()
engine = runtime.deserialize_cuda_engine(buf)

# create buffer
for binding in engine:
    size = trt.volume(engine.get_binding_shape(binding)) * engine.max_batch_size
    host_mem = cuda.pagelocked_empty(shape=[size],dtype=np.float32)
    cuda_mem = cuda.mem_alloc(host_mem.nbytes)

    bindings.append(int(cuda_mem))
    if engine.binding_is_input(binding):
        host_inputs.append(host_mem)
        cuda_inputs.append(cuda_mem)
    else:
        host_outputs.append(host_mem)
        cuda_outputs.append(cuda_mem)

...

# inference
start_time = time.time()
cuda.memcpy_htod_async(cuda_inputs[0], host_inputs[0], stream)
context.execute_async(bindings=bindings, stream_handle=stream.handle)
cuda.memcpy_dtoh_async(host_outputs[0], cuda_outputs[0], stream)
stream.synchronize()
print("execute times "+str(time.time()-start_time))

Thanks.

1 Like