Error Generating copyout(srcImg) [if not already present]', and 2)

Hello,

When I use nvc++ to compile code file with openACC, I got the error information:

#include <opencv2/opencv.hpp>
#include
#include
#include
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include<openacc.h>

using namespace std;
using namespace cv;

int main(){

 cv::Mat srcImg=cv::imread("/home/testSpace/images/blue-mountains.jpg");
 if(!srcImg.data){

 cout<<"The file is not loaded or does not exist"<<endl;
 return -1;

 }


 cout<<"Matrix"<<srcImg.rows<<" "<<srcImg.cols<<endl;

Mat duplicate(srcImg.rows,srcImg.cols, CV_8UC1,Scalar::all(255) );

int i,j=0;

#pragma acc enter data create(srcImg[0:srcImg.rows][0:srcImg.cols])
{

 #pragma acc update device(srcImg[:srcImg.rows][:srcImg.cols])
 #pragma acc parallel loop 
 for(i=0;i<srcImg.rows;i++){
   #pragma acc loop
   for(j=0;j<srcImg.cols;j++){


       srcImg.at<uchar>(i,j)=srcImg.at<uchar>(i,j)-1;

   }

 }

#pragma acc data copyout(srcImg[:srcImg.rows][:srcImg.cols])

}

cout<<(int)srcImg.at<uchar>(23,45)<<endl;

// return 0;

}

main:
40, Generating enter data create(srcImg)
45, Generating update device(srcImg)
Generating NVIDIA GPU code
45, #pragma acc loop gang /* blockIdx.x /
47, #pragma acc loop vector(128) /
threadIdx.x */
45, Generating implicit copyin(srcImg.step.p[:1],srcImg) [if not already present]
47, Loop is parallelizable
59, Generating copyout(srcImg) [if not already present]
T1 & cv::Mat::at(int, int):
1, include “opencv.hpp”
52, include “core.hpp”
59, include “mat.hpp”
3724, include “mat.inl.hpp”
1140, Generating implicit acc routine seq
Generating acc routine seq
Generating NVIDIA GPU code
cvflann::anyimpl::big_any_policy<std::__cxx11::basic_string<char, std::char_traits, std::allocator>>::print(std::basic_ostream<char, std::char_traits>&, void *const *):
1, include “opencv.hpp”
std::basic_ostream<T1, T2> & std::endl<char, std::char_traits>(std::basic_ostream<T1, T2> &):
1, include “opencv.hpp”
52, include “core.hpp”

My question is 1)why I got error information ’ Generating copyout(srcImg) [if not already present]', and 2) the result is Matrix810 1440 libgomp: TODO.

Hi glaciya2018,

2) the result is Matrix810 1440 libgomp: TODO.

I’m assuming you’re the same person who posted this question on stack overflow:

Same answer that I gave there in that you’ve most likely linked against the GNU OpenACC runtime (libgomp). Best guess is that it’s being brought in when you use ‘pgk-config’ to list the opencv2 libraries. I’d recommend you avoid doing this and only link with the libraries you need.

I also noticed that you added the “-nomp” flag. Presumably since we implicitly include our OpenMP runtime library, you were getting multiple defined references to the OpenMP runtime. If correct, then this is another sign that you’ve linked with libgomp. (GNU uses libgomp for both OpenMP and OpenACC).

1)why I got error information ’ Generating copyout(srcImg) [if not already present]’

Since you’re using unstructured data regions, the compiler needs to add the implicit copy clauses. Given OpenACC’s “present_or” semantics, at runtime the variable will be present and no data movement will occur.

However, you can remove the implicit copies by adding a “present” clause on the parallel loop, which tells the compiler to expect it to be present on the device. For example:

#pragma acc parallel loop present(srcImg)

Now this message may indicate another issue that you’ll encounter once you get your linking correct:

45, Generating implicit copyin(srcImg.step.p[:1],srcImg) [if not already present]

I don’t know the underlying structure of the “cv::Mat” type, but the compiler is seeing it as a class or struct. Hence my guess is that "srcImg[ ][ ]" may actually be an operator, and srcImg isn’t a 2D array. Hence how you’re using “create” and “update” may be problematic. Instead, you’d need to do a manual deep copy of the class in order to get the device copy to have the correct data layout.

Now given you probably don’t know the underlying data layout of a “cv:Mat”, it would be challenging to perform a deep copy to the device. I’d recommend you use CUDA Unified Memory by adding the “-gpu=managed” flag to you’re compilation and link flags. UM will have the CUDA driver implicitly manage data movement for all dynamically allocated data. Static objects still need to be managed via data directives, so you may still need to include “srcImg” in a data directive (without the brackets), but you wont need to manage the class’ underlying dynamic data.

-Mat

Hello Mat,

Many thanks for your help. 1)With your suggestion, I reviewed the code and found the image should be transformed into 2d matrix instead of 3d matrix.
2)And when I removed -nomp the error still comes out. I just have one doubt whether there is any problem on the openacc directive. I will make more tests .

Hello Mat,

After several tests, there is no luck in the result. with or without -nomp it doesn’t change the final output. And I made a test which is about adding pkg-config opencv4 --cflags --libs or without adding pkg-config opencv4 --cflags --libs. It turns out that without pkg-config opencv4 --cflags --libs the result can show the value of variables.

Based on test above,could I think whether I should use nvc++ or pgc++ to compile the opencv during installation? Because I took a look at the reference library of opencv, all of the library files point to gnu, I guess that the opencv is built by GCC or g++ by using GNU software.

I wish you could provide any suggestions.

This is the test I use with and without opencv lib

#include
#include<openacc.h>
#define N 5

using namespace std;

int main(){

int a[N];
int i,j=0;

#pragma acc data copyin(a[:N],i,j) //present(i,j)

{

    #pragma kernels present(a[:N],j,i)
{	
for(int i=0;i<N;i++){
      // #pragma acc loop //present(i,j)
	for(int j=0;j<N;j++){
             a[i]=i+j;         
                    
	}


      }
    }

#pragma acc data copyout(a[:N],j,i)
}

cout<<“a[N]”<<a[N-1]<<endl;
cout<<i<<endl;
cout<<j<<endl;

}

Thanks in advance.

You can, but it shouldn’t be necessary. We’re object compatible with GNU, so you should be able to just link in the OpenCV libraries.

Again, what I’m guess is happening is that when you run the pkg-config command, it’s returning more than just the OpenCV libraries, but also libgomp. Since this is put before our OpenACC runtime on the link line, the linker is resolving the OpenACC symbols from the GNU OpenACC runtime.

What I suggest you try is running the “pkg-config opencv4 --cflags --libs” by itself, then explicitly add just the needed libraries to your link line.

Can you please post the output from “pkg-config opencv4 --cflags --libs”? Again, I just guessing here, but if I can see what the output is, it would help to confirm my theory.

-Mat

Hello Mat,

Thanks for your reply.

Please take a look at the result by using “pkg-config opencv4 --cflags --libs” :

$ pkg-config opencv4 --cflags --libs

-I/usr/include/opencv4/opencv 
-I/usr/include/opencv4 
-lopencv_stitching 
-lopencv_aruco 
-lopencv_bgsegm 
-lopencv_bioinspired 
-lopencv_ccalib 
-lopencv_dnn_objdetect 
-lopencv_dnn_superres 
-lopencv_dpm 
-lopencv_highgui 
-lopencv_face 
-lopencv_freetype 
-lopencv_fuzzy 
-lopencv_hdf 
-lopencv_hfs 
-lopencv_img_hash 
-lopencv_line_descriptor 
-lopencv_quality 
-lopencv_reg 
-lopencv_rgbd 
-lopencv_saliency 
-lopencv_shape 
-lopencv_stereo 
-lopencv_structured_light 
-lopencv_phase_unwrapping 
-lopencv_superres 
-lopencv_optflow 
-lopencv_surface_matching 
-lopencv_tracking 
-lopencv_datasets 
-lopencv_text 
-lopencv_dnn 
-lopencv_plot 
-lopencv_ml 
-lopencv_videostab 
-lopencv_videoio 
-lopencv_viz 
-lopencv_ximgproc 
-lopencv_video 
-lopencv_xobjdetect 
-lopencv_objdetect 
-lopencv_calib3d 
-lopencv_imgcodecs 
-lopencv_features2d 
-lopencv_flann 
-lopencv_xphoto 
-lopencv_photo 
-lopencv_imgproc 
-lopencv_core

Thanks. I’m not seeing that the GOMP library is being added directly, but there may be a dependency in there someplace.

Does the output from the command “ldd <binary_name>” so any references to libgomp.so?

I’m not sure what it would entail, but you can try building OpenCV with nvc++.

Hello Mat,

Please take a look at the information below :

From the information:
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fba7d91e000

$ ldd testi_opencv

	linux-vdso.so.1 (0x00007ffded20f000)
	libopencv_stitching.so.4.2 => /lib/x86_64-linux-gnu/libopencv_stitching.so.4.2 (0x00007fba88cca000)
	libopencv_aruco.so.4.2 => /lib/x86_64-linux-gnu/libopencv_aruco.so.4.2 (0x00007fba88c5c000)
	libopencv_bgsegm.so.4.2 => /lib/x86_64-linux-gnu/libopencv_bgsegm.so.4.2 (0x00007fba88c37000)
	libopencv_bioinspired.so.4.2 => /lib/x86_64-linux-gnu/libopencv_bioinspired.so.4.2 (0x00007fba88bf5000)
	libopencv_ccalib.so.4.2 => /lib/x86_64-linux-gnu/libopencv_ccalib.so.4.2 (0x00007fba88b8a000)
	libopencv_dnn_objdetect.so.4.2 => /lib/x86_64-linux-gnu/libopencv_dnn_objdetect.so.4.2 (0x00007fba88b7d000)
	libopencv_dnn_superres.so.4.2 => /lib/x86_64-linux-gnu/libopencv_dnn_superres.so.4.2 (0x00007fba88b6c000)
	libopencv_dpm.so.4.2 => /lib/x86_64-linux-gnu/libopencv_dpm.so.4.2 (0x00007fba88b4f000)
	libopencv_highgui.so.4.2 => /lib/x86_64-linux-gnu/libopencv_highgui.so.4.2 (0x00007fba88b35000)
	libopencv_face.so.4.2 => /lib/x86_64-linux-gnu/libopencv_face.so.4.2 (0x00007fba88a96000)
	libopencv_freetype.so.4.2 => /lib/x86_64-linux-gnu/libopencv_freetype.so.4.2 (0x00007fba88a8b000)
	libopencv_fuzzy.so.4.2 => /lib/x86_64-linux-gnu/libopencv_fuzzy.so.4.2 (0x00007fba88a6a000)
	libopencv_hdf.so.4.2 => /lib/x86_64-linux-gnu/libopencv_hdf.so.4.2 (0x00007fba88a5c000)
	libopencv_hfs.so.4.2 => /lib/x86_64-linux-gnu/libopencv_hfs.so.4.2 (0x00007fba88a48000)
	libopencv_img_hash.so.4.2 => /lib/x86_64-linux-gnu/libopencv_img_hash.so.4.2 (0x00007fba88a2b000)
	libopencv_line_descriptor.so.4.2 => /lib/x86_64-linux-gnu/libopencv_line_descriptor.so.4.2 (0x00007fba889fc000)
	libopencv_quality.so.4.2 => /lib/x86_64-linux-gnu/libopencv_quality.so.4.2 (0x00007fba889e2000)
	libopencv_reg.so.4.2 => /lib/x86_64-linux-gnu/libopencv_reg.so.4.2 (0x00007fba889b5000)
	libopencv_rgbd.so.4.2 => /lib/x86_64-linux-gnu/libopencv_rgbd.so.4.2 (0x00007fba888fb000)
	libopencv_saliency.so.4.2 => /lib/x86_64-linux-gnu/libopencv_saliency.so.4.2 (0x00007fba888cb000)
	libopencv_shape.so.4.2 => /lib/x86_64-linux-gnu/libopencv_shape.so.4.2 (0x00007fba8889a000)
	libopencv_stereo.so.4.2 => /lib/x86_64-linux-gnu/libopencv_stereo.so.4.2 (0x00007fba8886d000)
	libopencv_structured_light.so.4.2 => /lib/x86_64-linux-gnu/libopencv_structured_light.so.4.2 (0x00007fba88852000)
	libopencv_phase_unwrapping.so.4.2 => /lib/x86_64-linux-gnu/libopencv_phase_unwrapping.so.4.2 (0x00007fba88845000)
	libopencv_superres.so.4.2 => /lib/x86_64-linux-gnu/libopencv_superres.so.4.2 (0x00007fba88818000)
	libopencv_optflow.so.4.2 => /lib/x86_64-linux-gnu/libopencv_optflow.so.4.2 (0x00007fba88795000)
	libopencv_surface_matching.so.4.2 => /lib/x86_64-linux-gnu/libopencv_surface_matching.so.4.2 (0x00007fba88743000)
	libopencv_tracking.so.4.2 => /lib/x86_64-linux-gnu/libopencv_tracking.so.4.2 (0x00007fba884f4000)
	libopencv_datasets.so.4.2 => /lib/x86_64-linux-gnu/libopencv_datasets.so.4.2 (0x00007fba8847b000)
	libopencv_text.so.4.2 => /lib/x86_64-linux-gnu/libopencv_text.so.4.2 (0x00007fba8840c000)
	libopencv_dnn.so.4.2 => /lib/x86_64-linux-gnu/libopencv_dnn.so.4.2 (0x00007fba880e5000)
	libopencv_plot.so.4.2 => /lib/x86_64-linux-gnu/libopencv_plot.so.4.2 (0x00007fba880d6000)
	libopencv_ml.so.4.2 => /lib/x86_64-linux-gnu/libopencv_ml.so.4.2 (0x00007fba88018000)
	libopencv_videostab.so.4.2 => /lib/x86_64-linux-gnu/libopencv_videostab.so.4.2 (0x00007fba87fbc000)
	libopencv_videoio.so.4.2 => /lib/x86_64-linux-gnu/libopencv_videoio.so.4.2 (0x00007fba87f3b000)
	libopencv_viz.so.4.2 => /lib/x86_64-linux-gnu/libopencv_viz.so.4.2 (0x00007fba87ece000)
	libopencv_ximgproc.so.4.2 => /lib/x86_64-linux-gnu/libopencv_ximgproc.so.4.2 (0x00007fba87c5e000)
	libopencv_video.so.4.2 => /lib/x86_64-linux-gnu/libopencv_video.so.4.2 (0x00007fba87be4000)
	libopencv_xobjdetect.so.4.2 => /lib/x86_64-linux-gnu/libopencv_xobjdetect.so.4.2 (0x00007fba87bca000)
	libopencv_objdetect.so.4.2 => /lib/x86_64-linux-gnu/libopencv_objdetect.so.4.2 (0x00007fba87b64000)
	libopencv_calib3d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_calib3d.so.4.2 (0x00007fba87991000)
	libopencv_imgcodecs.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgcodecs.so.4.2 (0x00007fba87936000)
	libopencv_features2d.so.4.2 => /lib/x86_64-linux-gnu/libopencv_features2d.so.4.2 (0x00007fba87862000)
	libopencv_flann.so.4.2 => /lib/x86_64-linux-gnu/libopencv_flann.so.4.2 (0x00007fba87803000)
	libopencv_xphoto.so.4.2 => /lib/x86_64-linux-gnu/libopencv_xphoto.so.4.2 (0x00007fba877b8000)
	libopencv_photo.so.4.2 => /lib/x86_64-linux-gnu/libopencv_photo.so.4.2 (0x00007fba876e5000)
	libopencv_imgproc.so.4.2 => /lib/x86_64-linux-gnu/libopencv_imgproc.so.4.2 (0x00007fba8713e000)
	libopencv_core.so.4.2 => /lib/x86_64-linux-gnu/libopencv_core.so.4.2 (0x00007fba86d8c000)
	libacchost.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libacchost.so (0x00007fba86b1b000)
	libaccdevaux.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevaux.so (0x00007fba86905000)
	libaccdevice.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevice.so (0x00007fba865d5000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fba865cf000)
	libcudadevice.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libcudadevice.so (0x00007fba863b8000)
	libatomic.so.1 => /lib/x86_64-linux-gnu/libatomic.so.1 (0x00007fba863ac000)
	libnvhpcatm.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libnvhpcatm.so (0x00007fba861a1000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fba85fbf000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fba85f9c000)
	libnvcpumath.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libnvcpumath.so (0x00007fba85b95000)
	libnvc.so => /opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libnvc.so (0x00007fba85932000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fba8573e000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fba85723000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fba855d4000)
	libgtk-3.so.0 => /lib/x86_64-linux-gnu/libgtk-3.so.0 (0x00007fba84e23000)
	libgdk-3.so.0 => /lib/x86_64-linux-gnu/libgdk-3.so.0 (0x00007fba84d1e000)
	libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007fba84bf9000)
	libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007fba84bd1000)
	libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fba84b71000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fba84a48000)
	libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007fba84989000)
	libharfbuzz.so.0 => /lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007fba84885000)
	libhdf5_serial.so.103 => /lib/x86_64-linux-gnu/libhdf5_serial.so.103 (0x00007fba84506000)
	libtesseract.so.4 => /lib/x86_64-linux-gnu/libtesseract.so.4 (0x00007fba841d8000)
	libprotobuf.so.17 => /lib/x86_64-linux-gnu/libprotobuf.so.17 (0x00007fba83ee0000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fba88dc5000)
	libdc1394.so.22 => /lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007fba83e66000)
	libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007fba83d1f000)
	libgstapp-1.0.so.0 => /lib/x86_64-linux-gnu/libgstapp-1.0.so.0 (0x00007fba83d0d000)
	libgstriff-1.0.so.0 => /lib/x86_64-linux-gnu/libgstriff-1.0.so.0 (0x00007fba83cfc000)
	libgstpbutils-1.0.so.0 => /lib/x86_64-linux-gnu/libgstpbutils-1.0.so.0 (0x00007fba83cbe000)
	libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007fba826fd000)
	libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007fba82481000)
	libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007fba82358000)
	libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007fba822c1000)
	libvtkInteractionStyle-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkInteractionStyle-6.3.so.6.3 (0x00007fba82231000)
	libvtkRenderingLOD-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingLOD-6.3.so.6.3 (0x00007fba82215000)
	libvtkIOPLY-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkIOPLY-6.3.so.6.3 (0x00007fba821f5000)
	libvtkIOGeometry-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkIOGeometry-6.3.so.6.3 (0x00007fba820c3000)
	libvtkFiltersTexture-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersTexture-6.3.so.6.3 (0x00007fba8209b000)
	libvtkIOExport-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkIOExport-6.3.so.6.3 (0x00007fba82009000)
	libvtkRenderingContextOpenGL-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingContextOpenGL-6.3.so.6.3 (0x00007fba81fd1000)
	libvtkRenderingOpenGL-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingOpenGL-6.3.so.6.3 (0x00007fba81d0a000)
	libvtkIOImage-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkIOImage-6.3.so.6.3 (0x00007fba81b56000)
	libvtkIOCore-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkIOCore-6.3.so.6.3 (0x00007fba81ad9000)
	libvtkRenderingFreeType-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingFreeType-6.3.so.6.3 (0x00007fba81a0c000)
	libvtkRenderingCore-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingCore-6.3.so.6.3 (0x00007fba8175b000)
	libvtkFiltersExtraction-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersExtraction-6.3.so.6.3 (0x00007fba8162a000)
	libvtkImagingCore-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkImagingCore-6.3.so.6.3 (0x00007fba81436000)
	libvtkFiltersSources-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersSources-6.3.so.6.3 (0x00007fba8136a000)
	libvtkFiltersGeneral-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersGeneral-6.3.so.6.3 (0x00007fba81002000)
	libvtkFiltersCore-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersCore-6.3.so.6.3 (0x00007fba80ad4000)
	libvtkCommonExecutionModel-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonExecutionModel-6.3.so.6.3 (0x00007fba80a22000)
	libvtkCommonDataModel-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonDataModel-6.3.so.6.3 (0x00007fba806a0000)
	libvtkCommonTransforms-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonTransforms-6.3.so.6.3 (0x00007fba8066e000)
	libvtkCommonMath-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonMath-6.3.so.6.3 (0x00007fba8064b000)
	libvtkCommonCore-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonCore-6.3.so.6.3 (0x00007fba80305000)
	libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007fba80280000)
	libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x00007fba80216000)
	libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007fba801de000)
	libgdcmMSFF.so.3.0 => /lib/x86_64-linux-gnu/libgdcmMSFF.so.3.0 (0x00007fba7ff38000)
	libtiff.so.5 => /lib/x86_64-linux-gnu/libtiff.so.5 (0x00007fba7feb5000)
	libIlmImf-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmImf-2_3.so.24 (0x00007fba7fbd6000)
	libgdal.so.26 => /lib/libgdal.so.26 (0x00007fba7e84f000)
	libgdcmDSED.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDSED.so.3.0 (0x00007fba7e71d000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fba7e701000)
	libtbb.so.2 => /lib/x86_64-linux-gnu/libtbb.so.2 (0x00007fba7e6b9000)
	libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fba7e6b3000)
	libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007fba7e6a1000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007fba7e564000)
	libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x00007fba7e552000)
	libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007fba7e548000)
	libcairo-gobject.so.2 => /lib/x86_64-linux-gnu/libcairo-gobject.so.2 (0x00007fba7e53c000)
	libatk-1.0.so.0 => /lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007fba7e512000)
	libatk-bridge-2.0.so.0 => /lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0 (0x00007fba7e4db000)
	libepoxy.so.0 => /lib/x86_64-linux-gnu/libepoxy.so.0 (0x00007fba7e3a8000)
	libfribidi.so.0 => /lib/x86_64-linux-gnu/libfribidi.so.0 (0x00007fba7e38b000)
	libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007fba7e1a8000)
	libpangoft2-1.0.so.0 => /lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007fba7e18f000)
	libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007fba7e140000)
	libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007fba7e0f9000)
	libXinerama.so.1 => /lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007fba7e0f4000)
	libXrandr.so.2 => /lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007fba7e0e5000)
	libXcursor.so.1 => /lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007fba7e0d8000)
	libXcomposite.so.1 => /lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007fba7e0d3000)
	libXdamage.so.1 => /lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007fba7e0ce000)
	libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007fba7e08c000)
	libwayland-cursor.so.0 => /lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007fba7e081000)
	libwayland-egl.so.1 => /lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007fba7e07c000)
	libwayland-client.so.0 => /lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007fba7e069000)
	libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007fba7e054000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fba7e04a000)
	libpixman-1.so.0 => /lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007fba7dfa3000)
	libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007fba7df9e000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fba7df72000)
	libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007fba7df63000)
	libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fba7dd59000)
	libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007fba7dd4d000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fba7dcda000)
	libgraphite2.so.3 => /lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007fba7dcad000)
	libsz.so.2 => /lib/x86_64-linux-gnu/libsz.so.2 (0x00007fba7dca6000)
	libarchive.so.13 => /lib/x86_64-linux-gnu/libarchive.so.13 (0x00007fba7dbe3000)
	liblept.so.5 => /lib/x86_64-linux-gnu/liblept.so.5 (0x00007fba7d960000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fba7d91e000)	libraw1394.so.11 => /lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007fba7d70f000)
	libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007fba7d6f3000)
	libgstbase-1.0.so.0 => /lib/x86_64-linux-gnu/libgstbase-1.0.so.0 (0x00007fba7d673000)
	libgstaudio-1.0.so.0 => /lib/x86_64-linux-gnu/libgstaudio-1.0.so.0 (0x00007fba7d5f6000)
	libgsttag-1.0.so.0 => /lib/x86_64-linux-gnu/libgsttag-1.0.so.0 (0x00007fba7d5b8000)
	libgstvideo-1.0.so.0 => /lib/x86_64-linux-gnu/libgstvideo-1.0.so.0 (0x00007fba7d507000)
	libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007fba7d4e5000)
	libvpx.so.6 => /lib/x86_64-linux-gnu/libvpx.so.6 (0x00007fba7d2ab000)
	libwebpmux.so.3 => /lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007fba7d29f000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fba7d276000)
	librsvg-2.so.2 => /lib/x86_64-linux-gnu/librsvg-2.so.2 (0x00007fba7c94b000)
	libzvbi.so.0 => /lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007fba7c8bc000)
	libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007fba7c8b1000)
	libaom.so.0 => /lib/x86_64-linux-gnu/libaom.so.0 (0x00007fba7c411000)
	libcodec2.so.0.9 => /lib/x86_64-linux-gnu/libcodec2.so.0.9 (0x00007fba7b62d000)
	libgsm.so.1 => /lib/x86_64-linux-gnu/libgsm.so.1 (0x00007fba7b61e000)
	libmp3lame.so.0 => /lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007fba7b5a6000)
	libopenjp2.so.7 => /lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007fba7b550000)
	libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007fba7b4f1000)
	libshine.so.3 => /lib/x86_64-linux-gnu/libshine.so.3 (0x00007fba7b2e4000)
	libspeex.so.1 => /lib/x86_64-linux-gnu/libspeex.so.1 (0x00007fba7b2c6000)
	libtheoraenc.so.1 => /lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007fba7b289000)
	libtheoradec.so.1 => /lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007fba7b269000)
	libtwolame.so.0 => /lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007fba7b243000)
	libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007fba7b215000)
	libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007fba7b168000)
	libwavpack.so.1 => /lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007fba7b13c000)
	libx264.so.155 => /lib/x86_64-linux-gnu/libx264.so.155 (0x00007fba7ae7e000)
	libx265.so.179 => /lib/x86_64-linux-gnu/libx265.so.179 (0x00007fba79f10000)
	libxvidcore.so.4 => /lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007fba79dfd000)
	libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007fba79dd4000)
	libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007fba79c18000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fba79c05000)
	libgme.so.0 => /lib/x86_64-linux-gnu/libgme.so.0 (0x00007fba79bb8000)
	libopenmpt.so.0 => /lib/x86_64-linux-gnu/libopenmpt.so.0 (0x00007fba799cb000)
	libchromaprint.so.1 => /lib/x86_64-linux-gnu/libchromaprint.so.1 (0x00007fba799b5000)
	libbluray.so.2 => /lib/x86_64-linux-gnu/libbluray.so.2 (0x00007fba79962000)
	libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007fba7978c000)
	libssh-gcrypt.so.4 => /lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007fba79700000)
	libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007fba796fb000)
	libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007fba796f3000)
	libvdpau.so.1 => /lib/x86_64-linux-gnu/libvdpau.so.1 (0x00007fba796ed000)
	libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007fba796d5000)
	libOpenCL.so.1 => /usr/local/cuda/targets/x86_64-linux/lib/libOpenCL.so.1 (0x00007fba794cd000)
	libvtkFiltersModeling-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersModeling-6.3.so.6.3 (0x00007fba7944f000)
	libvtkCommonMisc-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonMisc-6.3.so.6.3 (0x00007fba79436000)
	libvtkCommonSystem-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonSystem-6.3.so.6.3 (0x00007fba7941e000)
	libvtksys-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtksys-6.3.so.6.3 (0x00007fba793d0000)
	libvtkRenderingGL2PS-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingGL2PS-6.3.so.6.3 (0x00007fba793b2000)
	libvtkRenderingLabel-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingLabel-6.3.so.6.3 (0x00007fba792f9000)
	libgl2ps.so.1.4 => /lib/x86_64-linux-gnu/libgl2ps.so.1.4 (0x00007fba790e4000)
	libvtkRenderingContext2D-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkRenderingContext2D-6.3.so.6.3 (0x00007fba790a0000)
	libvtkFiltersGeometry-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersGeometry-6.3.so.6.3 (0x00007fba7900b000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007fba78f83000)
	libvtkImagingHybrid-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkImagingHybrid-6.3.so.6.3 (0x00007fba78efb000)
	libXt.so.6 => /lib/x86_64-linux-gnu/libXt.so.6 (0x00007fba78c92000)
	libvtkmetaio-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkmetaio-6.3.so.6.3 (0x00007fba78bf3000)
	libvtkDICOMParser-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkDICOMParser-6.3.so.6.3 (0x00007fba78bd6000)
	libvtkftgl-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkftgl-6.3.so.6.3 (0x00007fba78bc9000)
	libvtkCommonColor-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonColor-6.3.so.6.3 (0x00007fba78bac000)
	libvtkCommonComputationalGeometry-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkCommonComputationalGeometry-6.3.so.6.3 (0x00007fba78b6d000)
	libvtkFiltersStatistics-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkFiltersStatistics-6.3.so.6.3 (0x00007fba78ada000)
	libgdcmDICT.so.3.0 => /lib/x86_64-linux-gnu/libgdcmDICT.so.3.0 (0x00007fba78828000)
	libgdcmjpeg8.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg8.so.3.0 (0x00007fba787fe000)
	libgdcmjpeg12.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg12.so.3.0 (0x00007fba787d4000)
	libgdcmjpeg16.so.3.0 => /lib/x86_64-linux-gnu/libgdcmjpeg16.so.3.0 (0x00007fba787aa000)
	libCharLS.so.2 => /lib/x86_64-linux-gnu/libCharLS.so.2 (0x00007fba7875b000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fba78750000)
	libjson-c.so.4 => /lib/x86_64-linux-gnu/libjson-c.so.4 (0x00007fba7873e000)
	libgdcmIOD.so.3.0 => /lib/x86_64-linux-gnu/libgdcmIOD.so.3.0 (0x00007fba78726000)
	libgdcmCommon.so.3.0 => /lib/x86_64-linux-gnu/libgdcmCommon.so.3.0 (0x00007fba786fc000)
	libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007fba78653000)
	libjbig.so.0 => /lib/x86_64-linux-gnu/libjbig.so.0 (0x00007fba78445000)
	libHalf.so.24 => /lib/x86_64-linux-gnu/libHalf.so.24 (0x00007fba783fe000)
	libIex-2_3.so.24 => /lib/x86_64-linux-gnu/libIex-2_3.so.24 (0x00007fba783dd000)
	libIlmThread-2_3.so.24 => /lib/x86_64-linux-gnu/libIlmThread-2_3.so.24 (0x00007fba783d3000)
	libarmadillo.so.9 => /lib/libarmadillo.so.9 (0x00007fba783c0000)
	libpoppler.so.97 => /lib/x86_64-linux-gnu/libpoppler.so.97 (0x00007fba7807b000)
	libfreexl.so.1 => /lib/x86_64-linux-gnu/libfreexl.so.1 (0x00007fba78070000)
	libqhull.so.7 => /lib/x86_64-linux-gnu/libqhull.so.7 (0x00007fba77e14000)
	libgeos_c.so.1 => /lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007fba77dd1000)
	libepsilon.so.1 => /lib/x86_64-linux-gnu/libepsilon.so.1 (0x00007fba77db7000)
	libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x00007fba77d45000)
	libodbcinst.so.2 => /lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007fba77d2d000)
	libkmlbase.so.1 => /lib/x86_64-linux-gnu/libkmlbase.so.1 (0x00007fba77d10000)
	libkmldom.so.1 => /lib/x86_64-linux-gnu/libkmldom.so.1 (0x00007fba77c6d000)
	libkmlengine.so.1 => /lib/x86_64-linux-gnu/libkmlengine.so.1 (0x00007fba77c33000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fba77c05000)
	libxerces-c-3.2.so => /lib/x86_64-linux-gnu/libxerces-c-3.2.so (0x00007fba77872000)
	libnetcdf.so.15 => /lib/x86_64-linux-gnu/libnetcdf.so.15 (0x00007fba7774d000)
	libmfhdfalt.so.0 => /lib/libmfhdfalt.so.0 (0x00007fba77723000)
	libdfalt.so.0 => /lib/libdfalt.so.0 (0x00007fba7767c000)
	libogdi.so.4.1 => /lib/libogdi.so.4.1 (0x00007fba77660000)
	libgif.so.7 => /lib/x86_64-linux-gnu/libgif.so.7 (0x00007fba77655000)
	libgeotiff.so.5 => /lib/x86_64-linux-gnu/libgeotiff.so.5 (0x00007fba77620000)
	libcfitsio.so.8 => /lib/x86_64-linux-gnu/libcfitsio.so.8 (0x00007fba77321000)
	libpq.so.5 => /lib/x86_64-linux-gnu/libpq.so.5 (0x00007fba772d1000)
	libproj.so.15 => /lib/x86_64-linux-gnu/libproj.so.15 (0x00007fba76fe0000)
	libdapclient.so.6 => /lib/x86_64-linux-gnu/libdapclient.so.6 (0x00007fba76f98000)
	libdap.so.25 => /lib/x86_64-linux-gnu/libdap.so.25 (0x00007fba76dfa000)
	libspatialite.so.7 => /lib/x86_64-linux-gnu/libspatialite.so.7 (0x00007fba7686b000)
	libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007fba767dc000)
	libfyba.so.0 => /lib/x86_64-linux-gnu/libfyba.so.0 (0x00007fba76784000)
	libmysqlclient.so.21 => /lib/x86_64-linux-gnu/libmysqlclient.so.21 (0x00007fba7603b000)
	libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fba75d65000)
	libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fba75d14000)
	libatspi.so.0 => /lib/x86_64-linux-gnu/libatspi.so.0 (0x00007fba75cdd000)
	libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007fba75c7b000)
	libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fba75c50000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fba75c34000)
	libthai.so.0 => /lib/x86_64-linux-gnu/libthai.so.0 (0x00007fba75c29000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007fba75c23000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fba75c19000)
	libaec.so.0 => /lib/x86_64-linux-gnu/libaec.so.0 (0x00007fba75c10000)
	libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007fba75bd6000)
	libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007fba75bcb000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fba75baa000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fba75b7b000)
	liborc-0.4.so.0 => /lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007fba75af8000)
	libsoxr.so.0 => /lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007fba75a8d000)
	libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007fba75a80000)
	libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007fba75a71000)
	libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007fba7588b000)
	libmpg123.so.0 => /lib/x86_64-linux-gnu/libmpg123.so.0 (0x00007fba7582b000)
	libvorbisfile.so.3 => /lib/x86_64-linux-gnu/libvorbisfile.so.3 (0x00007fba75820000)
	libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fba756ea000)
	libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007fba756c7000)
	libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007fba75545000)
	libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007fba7552f000)
	libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007fba754f8000)
	libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fba75474000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007fba75356000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fba75331000)
	libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fba752e4000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007fba7522c000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007fba751f8000)
	libSM.so.6 => /lib/x86_64-linux-gnu/libSM.so.6 (0x00007fba751eb000)
	libICE.so.6 => /lib/x86_64-linux-gnu/libICE.so.6 (0x00007fba751cd000)
	libvtkImagingFourier-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkImagingFourier-6.3.so.6.3 (0x00007fba7519a000)
	libvtkalglib-6.3.so.6.3 => /lib/x86_64-linux-gnu/libvtkalglib-6.3.so.6.3 (0x00007fba75175000)
	libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007fba73774000)
	liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007fba71760000)
	libarpack.so.2 => /lib/x86_64-linux-gnu/libarpack.so.2 (0x00007fba71718000)
	libsuperlu.so.5 => /lib/x86_64-linux-gnu/libsuperlu.so.5 (0x00007fba716a8000)
	liblcms2.so.2 => /lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007fba7164d000)
	libnss3.so => /lib/x86_64-linux-gnu/libnss3.so (0x00007fba714fe000)
	libsmime3.so => /lib/x86_64-linux-gnu/libsmime3.so (0x00007fba714cc000)
	libnspr4.so => /lib/x86_64-linux-gnu/libnspr4.so (0x00007fba7148c000)
	libgeos-3.8.0.so => /lib/x86_64-linux-gnu/libgeos-3.8.0.so (0x00007fba712c3000)
	libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x00007fba712b8000)
	libminizip.so.1 => /lib/x86_64-linux-gnu/libminizip.so.1 (0x00007fba710ad000)
	liburiparser.so.1 => /lib/x86_64-linux-gnu/liburiparser.so.1 (0x00007fba7108c000)
	libhdf5_serial_hl.so.100 => /lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100 (0x00007fba71065000)
	libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fba70fd2000)
	libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007fba70f7c000)
	libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007fba70e53000)
	libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007fba70e28000)
	librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007fba70e08000)
	libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007fba70d9a000)
	libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007fba70d87000)
	liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007fba70d76000)
	libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007fba70d68000)
	libfyut.so.0 => /lib/x86_64-linux-gnu/libfyut.so.0 (0x00007fba70d5a000)
	libfygm.so.0 => /lib/x86_64-linux-gnu/libfygm.so.0 (0x00007fba70d51000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007fba70ca2000)
	libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007fba70c4b000)
	libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fba70bbb000)
	libdatrie.so.1 => /lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007fba70baf000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fba70b95000)
	libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007fba6f0d4000)
	libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fba6eff7000)
	libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fba6efc6000)
	libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fba6efbd000)
	libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fba6efae000)
	libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007fba6ece6000)
	libnssutil3.so => /lib/x86_64-linux-gnu/libnssutil3.so (0x00007fba6ecb3000)
	libplc4.so => /lib/x86_64-linux-gnu/libplc4.so (0x00007fba6ecac000)
	libplds4.so => /lib/x86_64-linux-gnu/libplds4.so (0x00007fba6eca5000)
	libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fba6ec88000)
	libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007fba6ec43000)
	libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007fba6ec20000)
	libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fba6ec17000)
	libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fba6ebcd000)
	libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007fba6ebc1000)
	libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007fba6eb2e000)
	libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007fba6ea87000)
	libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007fba6ea4d000)
	libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007fba6ea34000)
	libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007fba6ea0a000)
	libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007fba6e9f8000)
	libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007fba6e9aa000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fba6e96f000)

Lots of dependent libraries there. Not a big deal, but libgomp is definitely there. I have no idea as to why it’s getting in there, but presume there’s some dependency with one or more of the OpenCV libs.

This is long shot, but I’m wondering if you can use LD_PRELOAD to force the loader to load our runtime before libgomp. Something like:

LD_PRELOAD=/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libacchost.so:/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevaux.so:/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevice.so <your_exe_name> <args>

Otherwise, you may need to rebuild OpenCV with the NVHPC compilers, or possibly still with GNU if you can disable OpenMP. Though, I’ve not built OpenCV myself, so wont be much help here.

Hello Mat,

Thanks for your reply.

I tried to add LD_PRELOAD before exe file,but unfortunately, the issue still exists.

 $ LD_PRELOAD=/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libacchost.so:/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevaux.so:/opt/nvidia/hpc_sdk/Linux_x86_64/22.7/compilers/lib/libaccdevice.so ./testi_opencv

libgomp: TODO

Then I tried to load the environment of nvhpc by following your hints.
I used module loading to solve the issue:

export MODULEPATH=$MODULEPATH:/opt/nvidia/hpc_sdk/modulefiles
module load nvhpc


$ export MODULEPATH=$MODULEPATH:/opt/nvidia/hpc_sdk/modulefiles
$ module load nvhpc
$ sh pgcppi_opencv.sh
main:
     16, Generating copyin(a[:][:],temp) [if not already present]
     20, Generating present(a[:][:],temp)
     23, Loop is parallelizable
     25, Loop is parallelizable
         Generating NVIDIA GPU code
         23, #pragma acc loop gang, vector(128) collapse(2) /* blockIdx.x threadIdx.x */
         25,   /* blockIdx.x threadIdx.x auto-collapsed */
     43, Generating update self(temp,a[:])
$ ./testi_opencv
a[N][N]147
0


It works in the end.
1 Like