AK51
June 15, 2023, 5:56am
1
Hi,
I am looking at ActionNet
<img src="https://github.com/dusty-nv/jetson-inference/raw/master/docs/images/deep-vision-header.jpg" width="100%">
<p align="right"><sup><a href="posenet.md">Back</a> | <a href="backgroundnet.md">Next</a> | </sup><a href="../README.md#hello-ai-world"><sup>Contents</sup></a>
<br/>
<sup>Action Recognition</sup></s></p>
# Action Recognition
Action recognition classifies the activity, behavior, or gesture occuring over a sequence of video frames. The DNNs typically use image classification backbones with an added temporal dimension. For example, the ResNet18-based pre-trained models use a window of 16 frames. You can also skip frames to lengthen the window of time over which the model classifies actions.
<img src="https://github.com/dusty-nv/jetson-inference/raw/master/docs/images/actionnet-windsurfing.gif">
The [`actionNet`](../c/actionNet.h) object takes in one video frame at a time, buffers them as input to the model, and outputs the class with the highest confidence. [`actionNet`](../c/actionNet.h) can be used from [Python](https://rawgit.com/dusty-nv/jetson-inference/master/docs/html/python/jetson.inference.html#actionNet) and [C++](../c/actionNet.h).
As examples of using the `actionNet` class, there are sample programs for C++ and Python:
- [`actionnet.cpp`](../examples/actionnet/actionnet.cpp) (C++)
- [`actionnet.py`](../python/examples/actionnet.py) (Python)
## Running the Example
To run action recognition on a live camera stream or video, pass in a device or file path from the [Camera Streaming and Multimedia](aux-streaming.md) page.
show original
I have a scene that if someone do certain action, I want the Jetson board sends out a signal.
Thanks,
Hi @AK51 , these models for ActionNet came from https://github.com/kn1ghtf1re/Activity-Recognition-TensorRT and I haven’t trained them myself before. I believe that they in turn got the models from https://github.com/kenshohara/3D-ResNets-PyTorch or used those training scripts (I haven’t tried those though)
Those models use a ResNet backbone, but they have added modifications from an image classifier in order to incorporate the third temporal dimension, hence simply training an ImageNet/ResNet classifier won’t work.
Alternatively, TAO Toolkit supports training of your own action models and can be deployed with DeepStream:
system
July 12, 2023, 4:42am
5
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.