Multi-threaded code on Tx2

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi Folks,

I am looking to process videos from two cameras (IMX274) concurrently, using opencv APIs. I’ve implemented entire opencv/cuda processing pipeline as a c++ class. I would like to instantiate an object of the class for each camera. Furthermore, I would like to run each object of the class as an independent thread, which could be tied to one of 4 CPUs of Tx2. In this context -

  1. Is there an example of multithreaded code in Visionworks/Argus - which I can follow ?

  2. How could GPU be shared between two threads ? What care do I need to take to make sure that kernels launched by each of my C++ opencv processing object are thread safe ? Basically, how could GPU be shared between multiple contexts ? Is there an example that I can follow / learn from ?

  3. How can I make sure that underlying linux schedular does not interrupt my threads with any other tasks ? I.e. is there a way to ‘lock’ a thread on to a CPU - such that it would never get pre-empted, by linux ? This will help my thread - as its caches would not get thrashed at all, by preemption.

Thanks

2

Spawn a new thread. Don’t call into the same object from multiple threads at the same time:

pthread_create(...);

Create multiple contexts (one per thread) if you want to use GPU contexts or similar. If you need to process both of the images at the same time (say, calculating stereo separation) then you need to push them both into the same context, which must be run from a single thread.

Create thread attributes to specify when you create the thread, and set the SCHED_FIFO policy:

pthread_attr_setschedpolicy(..., SCHED_FIFO);

This requires root permissions, typically.

3

Thanks Snarky for your help.

I am looking to make use of 11_camera_object_identification code as template for aforesaid changes suggested by you.

When I attempt to run 11_camera_object_identification code - I do not see anything on screen. There are few similar question related to lack to screen graphics/images of 11_camera_object_identification code, on forum, but I could not infer the reason for not seeing the image.

Could you please help as to why 11_camera_object_identification code is now showing any output on Tx2 ?

I am at recently released r28.1.

Thanks

4

We don’t have samples demonstrating multi threads. Other users can give suggestion.

But it looks like you cannot run 11_camera_object_identification on r28.1, have you followed the steps to build/install caffe model?

5

Hi DaneLLL

THanks for your help.

Just to be sure, I am looking to follow the steps in README.md to build / install caffe model. I am running into some compilation issues after make -j4 … related to opencv

ubuntu@tegra-ubuntu:~/work/caffe/caffe$ vi Makefile.config
ubuntu@tegra-ubuntu:~/work/caffe/caffe$ make -j4
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/caffe.bin
.build_release/lib/libcaffe.so: undefined reference to `cv::String::allocate(unsigned long)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::deallocate()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/convert_imageset.bin' failed
make: *** [.build_release/tools/convert_imageset.bin] Error 1
make: *** Waiting for unfinished jobs....
.build_release/lib/libcaffe.so: undefined reference to `cv::String::allocate(unsigned long)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::deallocate()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/caffe.bin' failed
make: *** [.build_release/tools/caffe.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::String::allocate(unsigned long)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::deallocate()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/compute_image_mean.bin' failed
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::String::allocate(unsigned long)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::deallocate()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
collect2: error: ld returned 1 exit status
Makefile:625: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1

Then I modify Makefile.config to enable OPENCV_VERSION := 3

ubuntu@tegra-ubuntu:~/work/caffe/caffe$ vi Makefile.config

 OPENCV_VERSION := 3

After this I get following error

ubuntu@tegra-ubuntu:~/work/caffe/caffe$ make -j4
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/extract_features.bin
CXX/LD -o .build_release/tools/caffe.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/upgrade_solver_proto_text.bin
CXX/LD -o .build_release/tools/device_query.bin
CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin
CXX/LD -o .build_release/examples/mnist/convert_mnist_data.bin
CXX/LD -o .build_release/examples/cpp_classification/classification.bin
/usr/bin/ld: warning: libopencv_imgproc.so.3.2, needed by /usr/local/lib/libopencv_imgcodecs.so, may conflict with libopencv_imgproc.so.2.4
/usr/bin/ld: warning: libopencv_core.so.3.2, needed by /usr/local/lib/libopencv_imgcodecs.so, may conflict with libopencv_core.so.2.4
/usr/bin/ld: .build_release/examples/cpp_classification/classification.o: undefined reference to symbol '_ZN2cv6String10deallocateEv'
//usr/local/lib/libopencv_core.so.3.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:630: recipe for target '.build_release/examples/cpp_classification/classification.bin' failed
make: *** [.build_release/examples/cpp_classification/classification.bin] Error 1
make: *** Waiting for unfinished jobs....
ubuntu@tegra-ubuntu:~/work/caffe/caffe$
6

Hi,

MMAPI is implemented with Opencv4Tegra. Please build this sample with Opencv4Tegra.
Thanks.