Python3 Crash

Hello,

I’m trying to profile the following neural network: GitHub - AayushKrChaudhary/RITnet: This is a winning model of OpenEDS Semantic Segmentation Challenge

The network runs fine by itself. It also runs fine with NSight Systems. However, with NSight Compute, I get the following error:

*** Error in `/home/yingj4/anaconda3/bin/python3’: free(): invalid pointer: 0x00007fa64db211a0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fa67e0827e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fa67e08b37a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fa67e08f53c]

Machine details:
Ubuntu 16.04
NSight Compute 2019.5
NVIDIA Titan Xp
CUDA 10.2
Driver 440.36

I found a similar post, but it never received a resolution: nv-nsight-cu-cli + pytorch throws "free(): invalid pointer" error

Thank you!

This is an issue with Nsight Compute and pytorch > 19.07. It will be fixed in next Nsight Compute release.

Thank you for getting back to me. Are there any workarounds that we could use in the meantime? Are there any versions of NSight Compute and PyTorch that are known to work?

Also, is there an expected release date for the next version of NSight Compute?

Thank you!

From our experience, the issue only affects pytorch versions greater than 19.07. Other than that, there are no known workarounds.

We don’t provide release dates in this forum.