Gazenet Example Inference Result

Hello,

I’m trying to run the TLT Gazenet example.

I followed the Gazenet notebook instructions, I think it worked, I can train the model. However I was unable to get good results when running the inference.

I was using the pretrained tlt model and the provided MPIIFaceGaze subset. These are what I got:

I have tried different epoch_num, learning rate, but still can’t get good results.

Is it suppose to be like this, or did I miss anything?

Thank you so much.

Did you follow the jupyter notebook without any change? More, how about inference result with other images?

Hello,

Thanks for your reply.

I don’t think I changed anything in the notebook. But I got some warning messages while training.

And here are some other results:

Are above two images your own images or part of the MPIIFaceGaze dataset?

They are part of the MPIIFaceGaze dataset.

OK, I will try to reproduce your result. Thanks for the info.

Hello,

Do you have any update on this? Thank you

Sorry, I have not went through the gaznet training with jupyter notebook yet.
Could you please try to download the pretrained trainable version of gazenet model and run inference ( In step 7 Visualize Inference) again?
wget https://api.ngc.nvidia.com/v2/models/nvidia/tlt_gazenet/versions/trainable_v1.0/files/model.tlt

Or download the deployable version of gazenet model and run on the inference pipeline.

Hello,

If I understand correctly, do you mean skip the training and evaluation steps? I just did that and got an error ‘Please provide a valid gazenet file path for evaluation.’ Or is there anything need to be changed in the spec yaml file? Thank you.

Traceback (most recent call last):
File “/opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/driveix/build_wheel.runfiles/ai_infra/driveix/mitgazenet/scripts/inference.py”, line 121, in
File “/opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/driveix/build_wheel.runfiles/ai_infra/driveix/mitgazenet/scripts/inference.py”, line 117, in main
File “/opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/driveix/build_wheel.runfiles/ai_infra/driveix/mitgazenet/inferencer/gazenet_inferencer.py”, line 97, in build_model
ValueError: Please provide a valid gazenet file path for evaluation.
Traceback (most recent call last):
File “/usr/local/bin/gazenet”, line 8, in
sys.exit(main())
File “/opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/driveix/build_wheel.runfiles/ai_infra/driveix/mitgazenet/entrypoint/gazenet.py”, line 13, in main
File “/opt/tlt/.cache/dazel/_dazel_tlt/2b81a5aac84a1d3b7a324f2a7a6f400b/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/driveix/build_wheel.runfiles/ai_infra/driveix/common/entrypoint/entrypoint.py”, line 300, in launch_job
AssertionError: Process run failed.
2021-08-18 11:01:34,342 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.

Yes, skip the training and evaluation steps. Please use the pretrained trainable tlt model and directly run inference with it.
I try on my side, the inference result looks ok.
Please refer to below command in notebook.

# add by morganh
!tlt gazenet inference -e $SPECS_DIR/gazenet_tlt_pretrain.yaml
-i $DATA_DOWNLOAD_DIR/MPIIFaceGaze/sample-dataset/inference-set
-m /workspace/demo_3.0/public_data/tlt_cv_samples_v1.1.0/gazenet/pretrain_models/tlt_gazenet_vtrainable_v1.0/model.tlt
-o $USER_EXPERIMENT_DIR/experiment_result/exp1
-k $KEY





Hello,

I tried it again with the command:

!tlt gazenet inference -e $SPECS_DIR/gazenet_tlt_pretrain.yaml
-i $DATA_DOWNLOAD_DIR/MPIIFaceGaze/sample-dataset/inference-set
-m $LOCAL_EXPERIMENT_DIR/pretrain_models/tlt_gazenet_vtrainable_v1.0/model.tlt
-o $USER_EXPERIMENT_DIR/experiment_result/exp1
-k $KEY

But I still got an error ‘ValueError: Please provide a valid gazenet file path for evaluation.’

Please make sure $LOCAL_EXPERIMENT_DIR/pretrain_models/tlt_gazenet_vtrainable_v1.0/model.tlt is available.
You can run below to verify.
$ tlt unet run ls $LOCAL_EXPERIMENT_DIR/pretrain_models/tlt_gazenet_vtrainable_v1.0/model.tlt

Or you can directly run following command in terminal instead of notebook, to login in the docker.

$tlt unet run /bin/bash

Then , run above command to debug.

Hello,

I run the first command it gives me an error no such file or directory. I thought I followed the notebook to download it, can you tell me how to fix it? Thanks for your patience.

I tried again with:

!tlt gazenet inference -e $SPECS_DIR/gazenet_tlt_pretrain.yaml
-i $DATA_DOWNLOAD_DIR/MPIIFaceGaze/sample-dataset/inference-set
-m $USER_EXPERIMENT_DIR/pretrain_models/tlt_gazenet_vtrainable_v1.0/model.tlt
-o $USER_EXPERIMENT_DIR/experiment_result/exp1
-k $KEY

This time it works, thank you so much!