Is GazeNet supported in Deepstream 6.0.1

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform : Jetson
• DeepStream Version: 6.0.1
• Issue Type: question

I noticed that Deepstream 6.1 has some nice drawing of the gaze using the extra fields in :

typedef struct
{
  float gaze_params[cvcore::gazenet::GazeNet::OUTPUT_SIZE];
  int left_start_x;
  int left_start_y;
  int left_end_x;
  int left_end_y;
  int right_start_x;
  int right_start_y;
  int right_end_x;
  int right_end_y;
}NvDsGazeMetaData;

where as Deepstream 6.0.1’s version of that structure only has a:

typedef struct
{
  float gaze_params[cvcore::gazenet::GazeNet::OUTPUT_SIZE];
}NvDsGazeMetaData;

Q1: where can i find the documentation of the content for the gaze_params ?
Q2: should i be able to back-port the newer version of NvDsGazeMetaData to a deepstream 6.0.1 application ?

3

This is actually the output of gazenet, which is briefly described on NGC.

3D point of regards (X, Y, Z) and gaze vector (theta and phi)

About this model, you can see this link

Gaze Estimation - NVIDIA Docs.

I think this is not possible. DS-6.0 corresponds to the release/tao3.0_ds6.0ga branch, and the back-port may cause some problems.

4

Thank you @junshengy !!

ah, that matches

class CVCORE_API GazeNet
{
public:
    /**
     * Number of elements in network output.
     */
    static constexpr size_t OUTPUT_SIZE = 5;

I read here :

gaze_params[3] → theta (float): Gaze pitch in radians.
gaze_params[4] → phi (float): Gaze yaw in radians.

Q3: what is the 3D point of regard ? is that the base of the arrow in 1 of the eyes ? even though all the landmarks are in 2D ?

Q4: And am i correct that there is only 1 gaze arrow ? not 1 for each eye ?

5

Refer to the following link

If you want to draw arrows, you need additional parameters in version 6.1 or above. it’s 1 for each eye

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.