Opencv in Python Script Calling from C++

Hi,

Due to some reason, I am using Qt creator to develop GUI which would need to call a python script to handle image processing.

Before developing, I followed this https://jkjung-avt.github.io/opencv3-on-tx2/ to install opencv on TX2 board directly.
Below is the system information on my TX2 board:

  • Ubuntu 16.04 (directly on TX2)
  • Qt 3.5.1
  • Python 3.5m
  • Opencv 3.4.0
  • In the python script, the modules, numpy and opencv, are used.
    It is quite different of processing the python script directly and indirectly:

    1. The python script is processing successfully if I executed it directly by following command in console:
      python XXX.py
      
    2. However, the python script would be stuck in cv2 functions, such as cv2.imwrite(...) or cv2.normalize(...), if the python script is called by C++ in my Qt project.

    At first, I am wondering that is the performance bad in such way?
    The python script executes less than 5 minutes if it is called direct.
    But the python script looks like stuck in cv2 functions after it was called 1 hour ago.

    Is there anyone having the same issue?
    Or is there anything I missed?

    Hi,

    May I know how do you call the python function?
    Could you share your implementation with us for checking?

    By the way, OpenCV also have C++ support.
    It’s recommended to call the C++ API directly.

    Thanks.

    Hi Aasta,

    Thanks for your reply.

    Not only opencv is used in python script but also numpy is used.
    Since the python script is executed well in python, we consider not to move the part of opencv from the python script to C++ domain.

    Followings are my codes that how I call python function in C++:

    1. transform.cpp
      transform::transform(...) {
          setupPyFunc();
      }
      ...
      int transform::setupPyFunc()
      {
          Py_Initialize();
      
          if(!Py_IsInitialized())
              return ERROR_PYTHON_INIT_FAILED;
      
          // Relocate the working path of Python
          QString pyFuncPath = QString("sys.path.append(
      </li>
      <li> transform.h
      
      


      #include <python3.5m/Python.h>

      class transform: public QDockWidget
      {
      Q_OBJECT

      public:
      transform(…);
      ~transform();

      private:
      PyObject *mPyModule = nullptr;
      PyObject *mPyFuncDict= nullptr;
      PyObject *mPyFunc = nullptr;

      }

      
      </li>
      <li> processimage.py
      
      

      -- coding: utf-8 --


      import numpy as np
      import numpy.matlib
      import matplotlib.pyplot as plt
      import os
      from array import array
      import numpy.fft as fft
      import cv2
      import time
      from PIL import Image
      import re

      def main(processFolderName = None):
      if processFolderName is None:
      dirfolder = r’/home/nvidia/Workspace/project/v2-Debug/RawData0’
      else:
      dirfolder = r’/home/nvidia/Workspace/project/v2-Debug/'+processFolderName

      os.chdir(dirfolder)  # go to the dir
      dirs = os.listdir(dirfolder)  # list all the folders in dirfolder
      
      ...
                  img_SubBgr = np.round((DDD+ abs(np.min(DDD))) / abs(np.min(DDD)) * 255).astype('uint8')
                  img_DC = np.round((EEE+ abs(np.min(EEE))) / abs(np.min(EEE)) * 255).astype('uint8')
                  print("[Python] Debug Normalize")
      
                  # Normalize to [0, 255]
                  h, w = img_SubBgr.shape[:2]
                  img_SubBgr_norm = np.zeros((h, w)).astype('uint8')
                  print("[Python] Debug befor using cv2, version %s" % (cv2.__version__))
                  #<b>python hang in following function</b>
                  cv2.normalize(img_SubBgr,  img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)
                  ...
      
      
      </li>
      </ol>
      
      
      
      The tree graph of these files is as below:
      
      

      Project_Folder
      |–transform_folder
      | |–transform.cpp
      | |–transform.h
      | |–build_folder
      |–algorithm_folder
      |–processimage.py

      
      
      
      I also did run the executing file for testing overnight yesterday. 
      But it still stuck in the line "cv2.normalize(img_SubBgr,  img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)" when I was coming back to the office this morning.
      
      Is there anything I missed?
      
      I also tried to add "PyGILState_STATE  mPyGILState = PyGILState_Ensure()" and "PyGILState_Release(mPyGILState)" in handleImageProcess(...).
      But this is not work, either.
      
      
      Regards,
      C.T.")
                              + QApplication::applicationDirPath()
                              + QString("/../../transform_algorithm")
                              + QString("
      </li>
      <li> transform.h
      
      


      #include <python3.5m/Python.h>

      class transform: public QDockWidget
      {
      Q_OBJECT

      public:
      transform(…);
      ~transform();

      private:
      PyObject *mPyModule = nullptr;
      PyObject *mPyFuncDict= nullptr;
      PyObject *mPyFunc = nullptr;

      }

      
      </li>
      <li> processimage.py
      
      

      -- coding: utf-8 --


      import numpy as np
      import numpy.matlib
      import matplotlib.pyplot as plt
      import os
      from array import array
      import numpy.fft as fft
      import cv2
      import time
      from PIL import Image
      import re

      def main(processFolderName = None):
      if processFolderName is None:
      dirfolder = r’/home/nvidia/Workspace/project/v2-Debug/RawData0’
      else:
      dirfolder = r’/home/nvidia/Workspace/project/v2-Debug/'+processFolderName

      os.chdir(dirfolder)  # go to the dir
      dirs = os.listdir(dirfolder)  # list all the folders in dirfolder
      
      ...
                  img_SubBgr = np.round((DDD+ abs(np.min(DDD))) / abs(np.min(DDD)) * 255).astype('uint8')
                  img_DC = np.round((EEE+ abs(np.min(EEE))) / abs(np.min(EEE)) * 255).astype('uint8')
                  print("[Python] Debug Normalize")
      
                  # Normalize to [0, 255]
                  h, w = img_SubBgr.shape[:2]
                  img_SubBgr_norm = np.zeros((h, w)).astype('uint8')
                  print("[Python] Debug befor using cv2, version %s" % (cv2.__version__))
                  #<b>python hang in following function</b>
                  cv2.normalize(img_SubBgr,  img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)
                  ...
      
      
      </li>
      </ol>
      
      
      
      The tree graph of these files is as below:
      
      

      Project_Folder
      |–transform_folder
      | |–transform.cpp
      | |–transform.h
      | |–build_folder
      |–algorithm_folder
      |–processimage.py

      
      
      
      I also did run the executing file for testing overnight yesterday. 
      But it still stuck in the line "cv2.normalize(img_SubBgr,  img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)" when I was coming back to the office this morning.
      
      Is there anything I missed?
      
      I also tried to add "PyGILState_STATE  mPyGILState = PyGILState_Ensure()" and "PyGILState_Release(mPyGILState)" in handleImageProcess(...).
      But this is not work, either.
      
      
      Regards,
      C.T.)");
          qDebug() << "FuncPath: " << pyFuncPath;
          PyRun_SimpleString("import sys");
          PyRun_SimpleString("import numpy as np");
          PyRun_SimpleString("import numpy.matlib");
      
          PyRun_SimpleString("import matplotlib.pyplot as plt");
          PyRun_SimpleString("import os");
          PyRun_SimpleString("from array import array");
          PyRun_SimpleString("import numpy.fft as fft");
          PyRun_SimpleString("import cv2");
          PyRun_SimpleString("import time");
          PyRun_SimpleString("import re");
          PyRun_SimpleString("from PIL import Image");
      
          // Add current executed location into sys
          PyRun_SimpleString("sys.path.append('./')");
          PyRun_SimpleString(pyFuncPath.toLocal8Bit().data());
      
          //Load python module
          mPyModule = PyImport_ImportModule(PYTHON_MODULE_NAME);
          if (!mPyModule) {
              return ERROR_LOAD_MODULE_FAILED;
          }
          Py_INCREF(mPyModule);
      
          // Get function list from python module
          mPyFuncDict = PyModule_GetDict(mPyModule);
          if (!mPyFuncDict) {
              return ERROR_CANNOT_FIND_FUNC_DICTIONARY;
          }
          Py_INCREF(mPyFuncDict);
      
          // Load function
          mPyFunc = PyDict_GetItemString(mPyFuncDict, PYTHON_ALGORITHM_NAME);
          if (!mPyFunc || !PyCallable_Check(mPyFunc)) {
              return ERROR_LOAD_FUNCTION_FAILED;
          }
          Py_INCREF(mPyFunc);
          return 0;
      }
      ...
      
      void transform::receiveImage(...) {
          ...
          if (isProcess) {
              qDebug() << "Buff " << buffIndex << ": " << "Start imageProcess";
              QFuture<void> imageProcessFuture = QtConcurrent::run(this, &transform::handleImageProcess);
              imageProcessWatcher->setFuture(imageProcessFuture);
          }
      }
      
      void transform::handleImageProcess() {
          if(imageProcessed) {
              qDebug() << "ImageProcess: " << processFolderName;
      
              PyObject *pyResult = nullptr;
              // Necessary to convert QString to String while passing QString to Python
              PyObject *pyArgs = PyTuple_New(1);
              PyTuple_SetItem(pyArgs, 0, Py_BuildValue("s", processFolderName.toLocal8Bit().data()));
              Py_INCREF(pyArgs);
              pyResult = PyEval_CallObject(mPyFunc, pyArgs);
              Py_INCREF(pyResult);
              Py_DECREF(pyArgs);
              if (pyResult)
                  Py_DECREF(pyResult);
          }
      }
      
    2. transform.h
      ...
      #include <python3.5m/Python.h>
      
      class transform: public QDockWidget
      {
          Q_OBJECT
      
      public:
          transform(...);
          ~transform();
          ...
      private:
          PyObject *mPyModule = nullptr;
          PyObject *mPyFuncDict= nullptr;
          PyObject *mPyFunc = nullptr;
          ...
      }
      
    3. processimage.py
      # -*- coding: utf-8 -*-
      ...
      import numpy as np
      import numpy.matlib
      import matplotlib.pyplot as plt
      import os
      from array import array
      import numpy.fft as fft
      import cv2
      import time
      from PIL import Image
      import re
      
      ...
      
      def main(processFolderName = None):
          if processFolderName is None:
              dirfolder = r'/home/nvidia/Workspace/project/v2-Debug/RawData0'
          else:
              dirfolder = r'/home/nvidia/Workspace/project/v2-Debug/'+processFolderName
      
      
          os.chdir(dirfolder)  # go to the dir
          dirs = os.listdir(dirfolder)  # list all the folders in dirfolder
      
          ...
                      img_SubBgr = np.round((DDD+ abs(np.min(DDD))) / abs(np.min(DDD)) * 255).astype('uint8')
                      img_DC = np.round((EEE+ abs(np.min(EEE))) / abs(np.min(EEE)) * 255).astype('uint8')
                      print("[Python] Debug Normalize")
      
                      # Normalize to [0, 255]
                      h, w = img_SubBgr.shape[:2]
                      img_SubBgr_norm = np.zeros((h, w)).astype('uint8')
                      print("[Python] Debug befor using cv2, version %s" % (cv2.__version__))
                      #<b>python hang in following function</b>
                      cv2.normalize(img_SubBgr,  img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)
                      ...
      

    The tree graph of these files is as below:

    Project_Folder
        |--transform_folder
        |   |--transform.cpp
        |   |--transform.h
        |   |--build_folder
        |--algorithm_folder
            |--processimage.py
    

    I also did run the executing file for testing overnight yesterday.
    But it still stuck in the line “cv2.normalize(img_SubBgr, img_SubBgr_norm, 0, 255, cv2.NORM_MINMAX)” when I was coming back to the office this morning.

    Is there anything I missed?

    I also tried to add “PyGILState_STATE mPyGILState = PyGILState_Ensure()” and “PyGILState_Release(mPyGILState)” in handleImageProcess(…).
    But this is not work, either.

    Regards,
    C.T.

    Hi,

    What is DDD in your application?
    If DDD comes from C++, would you mind to update it to a local python parameter for testing?

    Thanks.

    have you found a solution for this issue?

    Hi Fmj99bx,

    We haven’t found a solution for this case yet.
    But currently, we don’t indirectly call the functions of openCV from python script but call them directly in Qt.

    Regards,
    C.T.

    Hi chavenal.tang, after 2 years i have the issue same to you. I can not call python script if i have cv2 function in my code ex ( cv2.resize, cv2.bitwise_and, …all function of cv2). Do you have some suggest for me about this?.
    Note: The same to you, i using qt-create for ui with c++ script and run image process stages with python script and return result to c++.
    I using opencv 4.5.5-dev, jetson xavier agx jetspack 4.5, Ubuntu 18.04, python 3.6m, Qt 5.9.5
    Rhe error message i get when call cv2 is:
    C free(): invalid pointer
    malloc(): memory corruption
    malloc.c:2401: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)’ failed.
    All errro occur when i call cv2 in python script, if i don’t call cv2 . Nothing happend.
    Thanks all to reading.