Jetson Nano - JetPack 4.6 - Non-OK-status: GpuLaunchKernel ... too many resources requested for launch

[pedro.marta]

Hi,

I’m currently using a Jetson Nano with the following spec:

• JetPack 4.6
• CUDA 10.2
• CudNN 8
• Tensorflow 2.5.0+nv21.7

Trying the following code sample returns an unexpected error:

import numpy as np
import tensorflow as tf

D = tf.convert_to_tensor(np.array([[1., 2., 3.], [-3., -7., -1.], [0., 5., -2.]]))
print(tf.linalg.det(D))

Error:

...
F tensorflow/core/kernels/linalg/determinant_op_gpu.cu.cc:102] Non-OK-status: GpuLaunchKernel( DeterminantFromPivotedLUKernel<Scalar, false>, config.block_count, config.thread_per_block, 0, device.stream(), config.virtual_thread_count, n, lu_factor.data(), pivots, nullptr, output.data()) status: Internal: too many resources requested for launch

Is there any known solution for the what seems to be an issue with the number of threads being used by each block?

Thanks for the attention!

[AastaLLL]

Hi,

This is a known issue for Nano.
Could you check if the latest v2.7.0+nv22.1 package also has this issue?

Thanks.

[pedro.marta]

Same outcome with that package:

2023-04-20 11:39:42.838564: F tensorflow/core/kernels/linalg/determinant_op_gpu.cu.cc:102] Non-OK-status: GpuLaunchKernel( DeterminantFromPivotedLUKernel<Scalar, false>, config.block_count, config.thread_per_block, 0, device.stream(), config.virtual_thread_count, n, lu_factor.data(), pivots, nullptr, output.data()) status: INTERNAL: too many resources requested for launch
Aborted (core dumped)

Aren’t there any other options like downgrading tensorflow or patching it and building from source?

Thanks for the help!

[pedro.marta]

Also, I’m not sure if it’s relevant information, but I’m running tensorflow inside a nvcr.io/nvidia/l4t-cuda:10.2.460-runtime docker container.

[AastaLLL]

Hi,

Could you try it without using a container?
Thanks.