I have a multi-threaded python script running on my TX2. The threads perform the following :
- infinite loop loads images (from file) and dumps them into a list (.append() ), S_thumbnails
- computes netvlad for each image. This is done with tensorflow, operation, GPU, takes about 150ms.
- pick 2 indices randomly [0, len(S_thumbnails)] and put in a queue, Q’
- consumes Q’
- Also references, S_thumbnails to do image operations using only CPU. S_thumbnails is a global variable
As this runs, I am monitoring the execution time of the tensorflow operation from thread1 and image processing operation from thread2. I observe that GPU slows down when using along with CPU, in this case GPU takes anywhere from 200ms to 500ms (fluctuating execution time). In other words, if I do not do any processing on thread2, gpu operations are faster.
I reckon this is a memory issue with race condition as S_thumbnails is used in both threads. How can I get past this issue? Any suggestions appreciated.
PS. Before i start my program I do as root: