Can't use gstreamer's nvjpegenc and opencv at the time

You may try using LD_PRELOAD so that libnvjpeg is found before libjpeg:

LD_PRELOAD=/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so ./your_app

If not working, a workaround would be to split the application into 2 processes, one using nvjpeg lib for encoding with nvjpegenc, and another one reading for using opencv or VPI such as:

Sender

gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! nvjpegenc ! image/jpeg,format=MJPG ! shmsink socket-path=jpg.sock

Receiver

gst-launch-1.0 shmsrc socket-path=jpg.sock ! jpegdec ! xvimagesink