When performing training with DetectNet_V2, the following error occurs
2022-05-16 23:55:46,404 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 14.235
2022-05-16 23:55:48,291 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 13.252
2022-05-16 23:55:50,121 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 13.666
2022-05-16 23:55:51,956 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 13.632
2022-05-16 23:55:53,851 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 13.196
2022-05-16 23:55:55,630 [INFO] modulus.hooks.sample_counter_hook: Train Samples / sec: 14.050
terminate called after throwing an instance of 'std::invalid_argument'
what(): Error decompressing JPEG image file /dataset/tlt-data/train/images/T9110014.jpg
[63355aec6841:02381] *** Process received signal ***
[63355aec6841:02381] Signal: Aborted (6)
[63355aec6841:02381] Signal code: (-6)
[63355aec6841:02381] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f29a82d1390]
[63355aec6841:02381] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f29a73d8438]
[63355aec6841:02381] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f29a73da03a]
[63355aec6841:02381] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x16d)[0x7f29a02e484d]
[63355aec6841:02381] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)[0x7f29a02e26b6]
[63355aec6841:02381] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)[0x7f29a02e2701]
[63355aec6841:02381] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8d38)[0x7f29a030dd38]
[63355aec6841:02381] [ 7] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f29a82c76ba]
[63355aec6841:02381] [ 8] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f29a74aa4dd]
[63355aec6841:02381] *** End of error message ***
/usr/local/bin/tlt-train: line 32: 2381 Aborted (core dumped) tlt-train-g1 ${PYTHON_ARGS[*]}
The command used is here.
tlt-train detectnet_v2 -e /workspace/tlt-experiments/detectnet/spec/detectnet_v2_train_resnet18_kitti.txt \
-r /workspace/tlt-experiments/detectnet/train_result \
-k $APIKEY \
-n detectnet
I think this error is caused by the image, because when I changed the train image, this error was resolved, but I do not know why the error occurred.
I would appreciate it if you could answer my questions.
• Hardware (RTX2080)
• Network Type (Detectnet_v2)
• TLT Version (TLT2.0)