Dear Friends,
I am using Jetson tx2 with jetpack 3.2 where tensorflow 1.7 is installed (from a gpu wheel). in regular codes where include print or monitor output, everything works fine. But when I disconnect monitor and remove all prints the code does not work! there are some errors related to cuda but it does not stop even when I wait for a long time. However when I put a single print, the code starts to work!!!
I realized this is related to tensorflow cause I only experienced this error when I running tensorflow part, but cant figure it out.
If you experienced, can you please help me in this regard?
I attached the photo of the error screen.
Many thanks
Hi,
Do you meet this error?
2018-02-23 16:45:13.345534: W ./tensorflow/core/common_runtime/gpu/pool_allocator.h:195] could not allocate pinned host memory of size: 267264.
2018-02-23 16:45:13.345585: E tensorflow/stream_executor/cuda/cuda_driver.cc:967] failed to alloc 240640 bytes on host: CUDA_ERROR_UNKNOWN.
2018-02-23 16:45:13.345634: W ./tensorflow/core/common_runtime/gpu/pool_allocator.h:195] could not allocate pinned host memory of size: 240640.
2018-02-23 16:45:13.345683: E tensorflow/stream_executor/cuda/cuda_driver.cc:967] failed to alloc 216576 bytes on host: CUDA_ERROR_UNKNOWN.
If yes, could you add following configure into your TensorFlow and give it a try?
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config, ...)
Thanks.
Yes, It worked by adding the following config :
device_count = {‘GPU’:0}
Strangely printing sess results is also solves the problem!!
Thanks for your help!
