Would u please help me?
I try to execute the sample codes (sampleMNIST, sampleOnnxMNIST) in order to compare the memory consumption under the fp32 and fp16 mode. The result is that both fp32 and fp16 mode almost take up the same memory.
I just compile the project, and use the command like this “./bin/sample_mnist” and “./bin/sample_mnist --fp16”, and then i type the "watch -n 0.1 -d nvidia-smi " in another terminal to record the GPU memory.
Is this situation normal? or maybe i missed some important details?
TensorRT Version: 22.214.171.124
GPU Type: Tesla V100-SXM2
Nvidia Driver Version: 410.79
CUDA Version: 10.0
CUDNN Version: 7.6.3
Operating System + Version: ubuntu 16.04