createNVBuffer camera read long run issue

lamegeorge · July 30, 2017, 1:16pm

Hi Guys,

I wish to read frames from camera continuously. I am trying to use createNVBuffer API to extract and store the frame. When I try to run the following code for a short duration it works without any issue. However, when I run it for a longer duration it I get the following error and the program terminates.

(Argus) Error InvalidState: (propagating from src/eglstream/FrameConsumerImpl.cpp, function releaseFrame(), line 317)
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 212)
size of uint8_t : 1
writeIndex : 2
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 315)

char *m_datamem ;
 while (true)
    {

        // Acquire a Frame.
        UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());

	cout<<"frame instantiated"<<endl;

        IFrame *iFrame = interface_cast<IFrame>(frame);

	cout<<"iFrame instantiated : "<<iFrame<< endl;

        if (!iFrame)
            break;

        // Get the Frame's Image.
        Image *image = iFrame->getImage();

        EGLStream::NV::IImageNativeBuffer *iImageNativeBuffer
              = interface_cast<EGLStream::NV::IImageNativeBuffer>(image);

        TEST_ERROR_RETURN(!iImageNativeBuffer, "Failed to create an IImageNativeBuffer");

	fd = iImageNativeBuffer->createNvBuffer(Argus::Size {m_framesize.width, m_framesize.height},
		       NvBufferColorFormat_YUV420, NvBufferLayout_Pitch, &status);

	if (status != STATUS_OK)
		       TEST_ERROR_RETURN(status != STATUS_OK, "Failed to create a native buffer");

        NvBufferParams params;
        NvBufferGetParams(fd, &params);
  
        int fsize = params.pitch[0] * m_framesize.height ;
    
        m_datamem = (char *)mmap(NULL, 1.5*fsize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, params.offset[0]);

         NvBufferDestroy(fd);

}

Kindly let me know what is wrong with this code. Is there a way to avoid allocating buffer at every iteration ? Is the destroy even happening ? Or is there some other issue.

Thanks.

ShaneCCC · July 31, 2017, 5:03am

@lamegeorge
Could you check if the sample code “/tegra_multimedia_api/samples/11_camera_object_identification” have the sample problem on your site and compare with it to figure out the problem.

lamegeorge · July 31, 2017, 6:47am

Hi ShaneCCC,

Thanks for your response. Based on your suggestion, I ran “/tegra_multimedia_api/samples/11_camera_object_identification” and got the following error :

(NvOdmDevice) Error NotInitialized: V4L2Device not powered on (in dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function setControlVal(), line 378)
(NvOdmDevice) Error NotInitialized: V4L2Device not powered on (in dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function setControlVal(), line 378)

Kindly let me know how to proceed.

Thanks.

lamegeorge · August 3, 2017, 11:26am

Hi ShaneCCC,

I am trying to follow camera object identification code specifically only the ConsumerThread code. I have copied the code from camera_caffe_main.cpp into the file where I was reading the frame from camera.

When I try to do the code gets compiled successfully but I get a Linking Error as below:

ci/libci.a(aaOCVConsumer.cpp.o): In function `ArgusSamples::OCVConsumerThread::threadExecute()':
aaOCVConsumer.cpp:(.text+0x5d0): undefined reference to `NvV4l2ElementPlane::qBuffer(v4l2_buffer&, NvBuffer*)'
aaOCVConsumer.cpp:(.text+0x6a8): undefined reference to `NvV4l2ElementPlane::waitForDQThread(unsigned int)'
ci/libci.a(aaOCVConsumer.cpp.o): In function `ArgusSamples::OCVConsumerThread::converterCapturePlaneDqCallback(v4l2_buffer*, NvBuffer*, NvBuffer*, void*)':
aaOCVConsumer.cpp:(.text+0x804): undefined reference to `NvV4l2ElementPlane::qBuffer(v4l2_buffer&, NvBuffer*)'
ci/libci.a(aaOCVConsumer.cpp.o): In function `ArgusSamples::OCVConsumerThread::createImageConverter()':
aaOCVConsumer.cpp:(.text+0xa24): undefined reference to `NvVideoConverter::createVideoConverter(char const*, int)'
aaOCVConsumer.cpp:(.text+0xb14): undefined reference to `NvV4l2ElementPlane::setDQThreadCallback(bool (*)(v4l2_buffer*, NvBuffer*, NvBuffer*, void*))'
aaOCVConsumer.cpp:(.text+0xb34): undefined reference to `NvV4l2ElementPlane::setDQThreadCallback(bool (*)(v4l2_buffer*, NvBuffer*, NvBuffer*, void*))'
aaOCVConsumer.cpp:(.text+0xb54): undefined reference to `NvVideoConverter::setOutputPlaneFormat(unsigned int, unsigned int, unsigned int, v4l2_nv_buffer_layout)'
aaOCVConsumer.cpp:(.text+0xc04): undefined reference to `NvVideoConverter::setCapturePlaneFormat(unsigned int, unsigned int, unsigned int, v4l2_nv_buffer_layout)'
aaOCVConsumer.cpp:(.text+0xcc0): undefined reference to `NvV4l2ElementPlane::setupPlane(v4l2_memory, unsigned int, bool, bool)'
aaOCVConsumer.cpp:(.text+0xd7c): undefined reference to `NvV4l2ElementPlane::setupPlane(v4l2_memory, unsigned int, bool, bool)'
aaOCVConsumer.cpp:(.text+0xe58): undefined reference to `NvV4l2ElementPlane::getNthBuffer(unsigned int)'
aaOCVConsumer.cpp:(.text+0xe90): undefined reference to `NvV4l2ElementPlane::setStreamStatus(bool)'
aaOCVConsumer.cpp:(.text+0xf34): undefined reference to `NvV4l2ElementPlane::setStreamStatus(bool)'
aaOCVConsumer.cpp:(.text+0xfd8): undefined reference to `NvV4l2ElementPlane::startDQThread(void*)'
aaOCVConsumer.cpp:(.text+0xfec): undefined reference to `NvV4l2ElementPlane::startDQThread(void*)'
aaOCVConsumer.cpp:(.text+0x1068): undefined reference to `NvV4l2ElementPlane::qBuffer(v4l2_buffer&, NvBuffer*)'
ci/libci.a(aaOCVConsumer.cpp.o): In function `ArgusSamples::OCVConsumerThread::abort()':
aaOCVConsumer.cpp:(.text+0x1164): undefined reference to `NvV4l2Element::abort()'
collect2: error: ld returned 1 exit status

My CMakeLists.txt looks like the following:

cmake_minimum_required(VERSION 2.8)
project( hardwareCuda )


#SET(CMAKE_C_COMPILER /usr/bin/gcc)
#SET(CMAKE_CXX_COMPILER /usr/bin/g++)

set(CMAKE_MODULE_PATH "/usr/lib/aarch64-linux-gnu/" "/home/ubuntu/Downloads/argus/cmake" "~/Downloads" "${CMAKE_MODULE_PATH}")

set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake" "${CMAKE_MODULE_PATH}")
set(CMAKE_C_FLAGS "-std=c++11")
#set(CMAKE_VERBOSE_MAKEFILE on)

find_package(PkgConfig REQUIRED)
pkg_check_modules(GTK3 REQUIRED gtk+-3.0)

find_package(Argus REQUIRED)
find_package(OpenGLES REQUIRED)
find_package(EGL REQUIRED)
find_package(X11 REQUIRED)
find_package( OpenCV  REQUIRED )
find_package( OpenVX  REQUIRED )
find_package(CUDA REQUIRED)
#find_package( VisionWorks REQUIRED )

include_directories( ${ARGUS_INCLUDE_DIR} ${EGL_INCLUDE_DIR} ${OPENGLES_INCLUDE_DIR} /home/ubuntu/Downloads/argus/samples/utils ${OpenCV_INCLUDE_DIRS} /usr/include /usr/local/include  /opt/intel/ipp/include         /home/ubuntu/tegra_multimedia_api/include/  ./ci /home/ubuntu/VisionWorks-1.6-Samples/nvxio/include/ /usr/local/cuda-8.0/targets/aarch64-linux/include/)

pkg_check_modules(GSTREAMER REQUIRED gstreamer-1.0)
include_directories(${GSTREAMER_INCLUDE_DIRS})

#set( CMAKE_VERBOSE_MAKEFILE on )

add_compile_options(-std=c++14)

cuda_add_executable( hardwareCuda aaTemplateMain.cpp absdiff.cu )

target_link_libraries( hardwareCuda ${OpenCV_LIBS}  ${ARGUS_LIBRARIES} ${OPENGLES_LIBRARIES} ${GSTREAMER_LIBRARIES} ${X11_LIBRARIES} ${EGL_LIBRARIES} ${GTK3_LIBRARIES} /usr/local/lib/libopencv_bgsegm.so.3.1.0 /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so /usr/lib/libvisionworks.so pthread /home/ubuntu/VisionWorks-1.6-Samples/libs/aarch64/linux/release/libovx.a  cuda )

Kindly let me know how to resolve this Linking error.

Thanks.

ShaneCCC · August 4, 2017, 5:48am

The NvV4l2ElementPlane and NvVideoConverter classes are in the samples/common/classes/ find the the define and build it to your app.

lamegeorge · August 4, 2017, 9:31am

Hi ShaneCCC,

Thanks for your response. I was able to find the required files at the location pointed by you. However, when I tried to build my application by supplying all the CPP files in this directory I seem to be getting the following message when I run my application. I have tried to adopt the same way of accessing camera as camera object identification code.

Failed to query video capabilities: Bad address
create vidoe converter return true
OCV CONSUMER: Waiting until producer is connected...
OCV CONSUMER: Producer has connected; continuing.
OCV CONSUMER: Done.

Any idea what could be going wrong ?

Thanks.

ShaneCCC · August 7, 2017, 4:30am

I think you need to figure below message output from and find the root cause.

Failed to query video capabilities: Bad address

lamegeorge · August 10, 2017, 2:09pm

Hi ShaneCCC,

I wish to process the raw YUV420 buffer directly and not make use of Video Converter. When I tried to study the code I observe that some APIs specific to Video Converter have been used. Can you suggest a way of following the sample in such a way that I can run the camera for longer duration before the process gets killed ?

Thanks.

lamegeorge · August 10, 2017, 3:03pm

Hi ShaneCCC,

I have the following queries :

What is the role of mutex lock and what is m_queueCond ? What resources is it safeguarding ?

pthread_mutex_lock(&m_queueLock);
        while (!m_gotError &&
            ((m_ConvOutputPlaneBufQueue->empty()) || (m_numPendingFrames >= MAX_PENDING_FRAMES)))
        {
            pthread_cond_wait(&m_queueCond, &m_queueLock);
        }

        if (m_gotError)
        {
            pthread_mutex_unlock(&m_queueLock);
            break;
        }

What is the trigger condition for the following two callbacks ?

m_ImageConverter->capture_plane.
        setDQThreadCallback(converterCapturePlaneDqCallback);

    m_ImageConverter->output_plane.
        setDQThreadCallback(converterOutputPlaneDqCallback);

If we are processing the frame as it comes and it takes more than (1/fps) time , what happens in this event ?
How to free the frame which has been received in case of the event described in #3. Does it create any problems ?

Kindly help me out.

Thanks.