Hi,
I’ve installed JetPack 2.3.1 (so, CUDA v8) on TX1, and tested one branch of ‘Caffe’, and got an error like below:
[ RUN ] ThresholdLayerTest/2.Test2
F0201 01:46:34.812922 8653 threshold_layer.cu:24] Check failed: error == cudaSuccess (8 vs. 0) invalid device function
*** Check failure stack trace: ***
@ 0x7fb3346718 google::LogMessage::Fail()
@ 0x7fb3348614 google::LogMessage::SendToLog()
@ 0x7fb3346290 google::LogMessage::Flush()
@ 0x7fb3348eb4 google::LogMessageFatal::~LogMessageFatal()
@ 0x7fb24bb8f8 caffe::ThresholdLayer<>::Forward_gpu()
@ 0x54a714 caffe::Layer<>::Forward()
@ 0x5929c0 caffe::ThresholdLayerTest_Test2_Test<>::TestBody()
@ 0xa58a74 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0xa51d1c testing::Test::Run()
@ 0xa51e58 testing::TestInfo::Run()
@ 0xa51f18 testing::TestCase::Run()
@ 0xa53078 testing::internal::UnitTestImpl::RunAllTests()
@ 0xa5338c testing::UnitTest::Run()
@ 0x53b5d8 main
@ 0x7fb1ef38a0 __libc_start_main
Makefile:526: recipe for target ‘runtest’ failed
make: *** [runtest] Aborted
I guess it is because of 'CUDA_ARCH’definition on Makefile.config of my Caffe branch:
CUDA_ARCH := -gencode arch=compute_20,code=sm_20
-gencode arch=compute_20,code=sm_21
-gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35
-gencode arch=compute_50,code=sm_50
-gencode arch=compute_52,code=sm_52
-gencode arch=compute_61,code=sm_61
Could 52 and 61 make problem on TX1 ?
Where can I find exact version for TX1 and other NVIDIA GPUs?
BR,
Jaebeom Park
Hello, are you sure you mean TK1? Maybe you actually meant TX1 since CUDA 8 isn’t supported on TK1.
For TX1, it appears the CUDA gencode is missing compute_53/sm_53 (for TX1’s GM20B GPU). Can you try this instead?
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
-gencode arch=compute_20,code=sm_21 \
-gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_53,code=sm_53 \
-gencode arch=compute_61,code=sm_61
Note the addition of -gencode arch=compute_53,code=sm_53 to the flags.
Sorry for confusion, TX1 is correct. I will try and leave comment soon. I’ve corrected my question for further convenience, thanks!
Thank you, dusty_nv! It works fine.
Dear sir
When I do make runtest,caffe shows the message like below:
F1221 10:25:21.685459 9173 syncedmem.cpp:57] Check failed: error == cudaSuccess (30 vs. 0) unknown error
*** Check failure stack trace: ***
@ 0x7f82f07d95cd google::LogMessage::Fail()
@ 0x7f82f07db433 google::LogMessage::SendToLog()
@ 0x7f82f07d915b google::LogMessage::Flush()
@ 0x7f82f07dbe1e google::LogMessageFatal::~LogMessageFatal()
@ 0x7f82eec22ce8 caffe::SyncedMemory::gpu_data()
@ 0x7f82eec69612 caffe::Blob<>::gpu_data()
@ 0x7f82eecaab4c caffe::ConvolutionLayer<>::Forward_gpu()
@ 0x45bcf2 caffe::Layer<>::Forward()
@ 0x45ec74 caffe::GradientChecker<>::CheckGradientSingle()
@ 0x463633 caffe::GradientChecker<>::CheckGradientExhaustive()
@ 0x752d5f caffe::ConvolutionLayerTest_Test1x1Gradient_Test<>::TestBody()
@ 0x7c86d3 testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x7c1cea testing::Test::Run()
@ 0x7c1e38 testing::TestInfo::Run()
@ 0x7c1f15 testing::TestCase::Run()
@ 0x7c31ef testing::internal::UnitTestImpl::RunAllTests()
@ 0x7c3513 testing::UnitTest::Run()
@ 0x451059 main
@ 0x7f82eded6830 __libc_start_main
@ 0x457259 _start
@ (nil) (unknown)
Makefile:476: recipe for target ‘runtest’ failed
make: *** [runtest] Aborted (core dumped)
Please help me how to solve this problem…
Hi,
We do find that not all Caffe run test passed on our platform.
A possible reason is that Caffe run test is designed for x86 machine and yield some difference on the ARM.
Although not all test passed, we don’t find any critical issue when inferencing with Caffe.
It’s recommended to give it a try and update information to us once you meet error in the real use case.
By the way, we also recommend using our TensorRT library for better performance.
Thanks.