How to run action recognition with python pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

**• Hardware Platform -------> GPU
**• DeepStream Version ------> 7.0
• TensorRT Version ------> 8.6.2.0
**• NVIDIA GPU Driver Version -----> 555

We are currently migrating a DeepStream-based Action Recognition application from C++ to Python using DeepStream 7.0. While the C++ version works correctly and we can access the NVDS_PREPROCESS_BATCH_META from the batch_user_meta_list, we’re unable to replicate this in Python due to apparent missing bindings in pyds.

{
    user_meta = (NvDsUserMeta *)(l_user_meta->data);
    if (user_meta->base_meta.meta_type == NVDS_PREPROCESS_BATCH_META)
    {
      GstNvDsPreProcessBatchMeta *preprocess_batchmeta =
          (GstNvDsPreProcessBatchMeta *)(user_meta->user_meta_data);
      std::string model_dims = "";
      if (preprocess_batchmeta->tensor_meta) {
        if (preprocess_batchmeta->tensor_meta->tensor_shape.size() == MODEL_3D_SHAPES) {
          model_dims = "3D: AR - ";
        } else {
          model_dims = "2D: AR - ";
        }

When we are trying to do the same thing in python

while l_user_meta is not None:
        try:
            user_meta = pyds.NvDsUserMeta.cast(l_user_meta.data)
        except StopIteration:
            break

        if user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDS_PREPROCESS_BATCH_META:
            preprocess_batchmeta = pyds.NvDsPreProcessBatchMeta.cast(
                user_meta.user_meta_data)
            
            print(preprocess_batchmeta)
            # -----------------------------------------------------------------
            # Derive the “2D/3D : AR – ” prefix once per preprocess batch
            # -----------------------------------------------------------------
            model_dims = ""
            if preprocess_batchmeta.tensor_meta:
                if preprocess_batchmeta.tensor_meta.tensor_shape.size() == MODEL_3D_SHAPES:
                    model_dims = "3D: AR - "
                else:
                    model_dims = "2D: AR - "

ERROR :- AttributeError: type object ‘pyds.NvDsMetaType’ has no attribute ‘NVDS_PREPROCESS_BATCH_META’

Our Analysis:

  • The enum NVDS_PREPROCESS_BATCH_META appears to be missing from PyDS bindings (Python API).
  • Other meta types (e.g., NVDSINFER_TENSOR_OUTPUT_META) are accessible, so the base functionality is working.
  • The struct GstNvDsPreProcessBatchMeta also does not seem to be exposed via PyDS.

Our Goal:

We need to:

  1. Access NVDS_PREPROCESS_BATCH_META and cast to GstNvDsPreProcessBatchMeta equivalent in Python.
  2. Parse ROI metadata and tensor shape (needed to distinguish 2D vs 3D models).
  3. Display per-ROI text using the classification or detection results based on the preprocess metadata.

Questions:

  1. Is NVDS_PREPROCESS_BATCH_META exposed in Python for DeepStream 7.0?
  2. If not, is there a workaround or plan to expose it now?
  3. Can you share an example of how to extract and use preprocess plugin metadata in Python?

If this functionality is not yet available in Python, we would appreciate:

A patch or workaround to extract this metadata
Or access to native C/C++ helper functions via Python C-bindings
3

Could you refer to this 316815? The related patches have been uploaded on that forum.

4

@yuweiw
How I can run this in deepstream-7.0 ?

5

You can migrate the patches to the DeepStream 7.0 yourself. We suggest that you upgrade the version of DeepStream to 7.1.

6

HI @yuweiw
In this topic:- Query regarding nvds meta_type - #16 by ereshmittal5

I followed your approach and updated the DeepStream Python bindings in the following directories:
📁 bindings/docstrings/

Added:

    nvdspreprocessmetadoc.h

    nvdsroimetadoc.h

Modified:

    nvdsmetadoc.h

📁 bindings/src/

bindnvds_roi_meta.cpp

bindnvdspreprocess_meta.cpp

bindnvdsmeta.cpp (extended)

📁 bindings/include/bind/

bindnvds_roi_meta.hpp

bindnvdspreprocess_meta.hpp

bindnvdsmeta.hpp (extended)

Other changes:

utils.hpp: Helpers for casting and metadata handling

pyds.cpp: Registered new bindings

docs/bindings_file_list.txt: Added the new files to the build list

What I’m Trying To Do

Build the .whl (Python wheel) using:

cd deepstream_python_apps/bindings
mkdir build && cd build
cmake ..
make

The Problem

While building, I’m getting multiple import or type reference errors during make. Examples:

Symbols not recognized from new .cpp or .hpp files

Unresolved references related to NvDsPreProcessBatchMeta

Some errors appear to be related to incorrect binding registrations or missing includes

Attachments

I’ve attached:

binding_files.zip (4.5 KB) (This is above topic we got )

deepstream_python_apps_action_changes.zip (7.7 MB) ( we make changes )

What I Need Help With

Can you please review if the changes are correct to extend pyds with new metadata types?

What are the required steps to correctly register new bindings in DeepStream 7.0's pyds?

How do I avoid or fix multiple symbol errors during the build?

Could you share a clean example or template for how NVIDIA recommends adding new metadata bindings?

Additional Notes

DeepStream Version: 7.0

Installed from: /opt/nvidia/deepstream/deepstream-7.0

Using existing v1.1.11 of pyds as base

I confirmed changes were applied cleanly; now just stuck during the final build steps

Any guidance, validation, or examples you could provide would be immensely helpful to unblock this.

Thanks so much in advance!

Best regards,
Debjit

7

Did you add the library paths to bindings/CMakeLists.txt Line 73? Also run docs/parse_bindings.py after updating bindings_file_list.txt and then try to build the wheel.

8

If you have any unclear details, please refer directly to this patch.

9

All code are updated to 7.1 right ? after that the respected person changed (added the code ). But there are lot’s of dependency file pyds.cpp, utils.cpp, utils.hpp and lot of other files got updated,
I am not able to understand it properly how I need to add how much not !

Could you please help me with 7.0 and I can run 3D action_recognation.py
please share me respective changes !

I am requesting you to do it !

10

We have provided the whole patch source code. All you need to do is to merge the source code into your own code, recompile the pyds binding, and reinstall it. This patch is compatible with version 7.0.

  1. Please learn the basic knowledge of pyds binding by referring to our Guide.

  2. Merge the source code from this patch into your own code. You can ignore the files of the following two folders: dist and pyds.egg-info.

  3. Rebuild the pyds and reinstall the generated packet by referring to our README.

11

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.