"Permission Denied" when launching the heartratenet notebook

• Issue Description

Hello,

I have followed this tutorial from Nvidia to train and use HeartRateNet:

TAO Toolkit Quickstart

After, opening Jupyter notebook with this command:

jupyter notebook --ip=0.0.0.0 --no-browser --allow-root

I saw that as output, i haven’t given a URL with ip address of “0.0.0.0” as seen in the tutorial. Instead the output gave me two URLs:

one with “127.0.0.1” and another with “jetsonnano”.

After opening the browser with the URL with “jetsonnano”, i have found and opened the notebook “heartratenet.ipynb”. As i have opened the notebook, i saw following lines on the terminal:

[I 13:11:26.717 NotebookApp] 302 GET /?token=************************************************ (127.0.0.1) 2.130000ms
[I 13:12:28.066 NotebookApp] Writing notebook-signing key to /home/User/.local/share/jupyter/notebook_secret
[W 13:12:28.073 NotebookApp] Notebook heartratenet/heartratenet.ipynb is not trusted
[I 13:12:33.379 NotebookApp] Kernel started: 7260a7ef-a39c-45f1-9cff-1b688f2cd965, name: python3
[W 13:12:33.510 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20231003131032 (127.0.0.1) 65.270000ms referer=http://jetsonnano:8888/notebooks/heartratenet/heartratenet.ipynb
[W 13:12:55.655 NotebookApp] Notebook heartratenet/heartratenet.ipynb is not trusted

Then, from the browser i have clicked on the button reads something like “not trusted” to make it trust the notebook and after few seconds the text in the button turned to “trusted”. As it turns to trusted, i have seen these lines on my terminal:

[W 13:12:55.730 NotebookApp] Trusting notebook /heartratenet/heartratenet.ipynb
[I 13:12:57.423 NotebookApp] Starting buffering for 7260a7ef-a39c-45f1-9cff-1b688f2cd965:f87b39e513e64cedb85d86894420f290
[W 13:12:58.894 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20231003131032 (127.0.0.1) 7.630000ms referer=http://jetsonnano:8888/notebooks/heartratenet/heartratenet.ipynb

After that, i have run the notebook by clicking on “Cell” and then on “Run All”, just as shown in the tutorial.

Then, i saw as the output of the “!mkdir -p $LOCAL_EXPERIMENT_DIR/model/” command which says permission denied.

Thanks in advance for all the help you offer.

P.S. I just begun to use the TAO toolkit (except for tao-converter) and i am a complete beginner on TAO, docker and jupyter notebooks. So even if it’s something simple, please explain.

• System Specs

• Hardware: Jetson Nano

• Network Type: HeartRateNet

• TAO Version: 4.0.1 (I have shared complete output of “!tao info --verbose” under “Notes” section below.)

• Training spec file: Please see “Notes” section below. (I have shared the complete output of "!cat $LOCAL_TRAIN_SPEC under “Notes” section below.)

• Jetpack Version: 4.6.4

• Python Version: 3.6.9

• How to reproduce the issue ?

  1. Open an NGC account and get the API key

  2. Enter the command:

sudo docker login nvcr.io

Then enter “$oauthtoken” to username and the API key as password.

  1. Enter these commands one by one:

$pip3 install virtualenv

$pip3 install virtualenvwrapper

$export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3

$export WORKON_HOME=~/Envs

$source ~/.local/bin/virtualenvwrapper.sh

$mkvirtualenv launcher

$deactivate

$wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.1/zip -O cv_samples_v1.4.1.zip
--2023-10-03 11:47:36--  https://api.ngc.nvidia.com/v2/resources/nvidia/tao/cv_samples/versions/v1.4.1/zip

$unzip -u cv_samples_v1.4.1.zip -d ./cv_samples_v1.4.1 && rm -rf cv_samples_v1.4.1.zip && cd ./cv_samples_v1.4.1

$workon launcher

$pip3 install notebook

$jupyter notebook --ip=0.0.0.0 --no-browser --allow-root

  1. See that you haven’t received a URL with “0.0.0.0” as seen in the guide and instead you have received two URLs one with “127.0.0.1” and “jetsonnano”.

  2. Right click on the URL with “jetsonnano” and select “Open Link”.

  3. As you see a list of folders with names of models on the browser navigate to “heartratenet” folder and open the notebook there.

  4. As you open the notebook notice the log with “404 GET” and the one reads “Notebook heartratenet/heartratenet.ipynb is not trusted” on the terminal.

  5. Set the notebook as trusted by clicking a button reads something like “not trusted” on the jupyter notebook page in the browser.

  6. Notice the log reads “Trusting notebook /heartratenet/heartratenet.ipynb” and the one with “404 GET”.

  7. Click on “Cell” and select “Run All”.

  8. Move to the first code block (under “0. Set up env variables, map drives and install dependencies”) and at the output of the section see permision failed at the end of output for “!mkdir -p $LOCAL_EXPERIMENT_DIR/model/”.

• Notes:

Here is the complete output of the “!tao info --verbose” command:

Configuration of the TAO Toolkit Instance

dockers: 		
	nvidia/tao/tao-toolkit: 			
		4.0.0-tf2.9.1: 				
			docker_registry: nvcr.io
			tasks: 
				1. classification_tf2
				2. efficientdet_tf2
		4.0.0-tf1.15.5: 				
			docker_registry: nvcr.io
			tasks: 
				1. augment
				2. bpnet
				3. classification_tf1
				4. detectnet_v2
				5. dssd
				6. emotionnet
				7. efficientdet_tf1
				8. faster_rcnn
				9. fpenet
				10. gazenet
				11. gesturenet
				12. heartratenet
				13. lprnet
				14. mask_rcnn
				15. multitask_classification
				16. retinanet
				17. ssd
				18. unet
				19. yolo_v3
				20. yolo_v4
				21. yolo_v4_tiny
				22. converter
		4.0.1-tf1.15.5: 				
			docker_registry: nvcr.io
			tasks: 
				1. mask_rcnn
				2. unet
		4.0.0-pyt: 				
			docker_registry: nvcr.io
			tasks: 
				1. action_recognition
				2. deformable_detr
				3. segformer
				4. re_identification
				5. pointpillars
				6. pose_classification
				7. n_gram
				8. speech_to_text
				9. speech_to_text_citrinet
				10. speech_to_text_conformer
				11. spectro_gen
				12. vocoder
				13. text_classification
				14. question_answering
				15. token_classification
				16. intent_slot_classification
				17. punctuation_and_capitalization
format_version: 2.0
toolkit_version: 4.0.1
published_date: 03/06/2023

Here is the full output of the “!cat $LOCAL_TRAIN_SPEC” command:

__class_name__: HeartRateNetTrainer
checkpoint_dir: /workspace/tao-experiments/heartratenet/model/
results_dir: /workspace/tao-experiments/heartratenet/
random_seed: 32
log_every_n_secs: 20
checkpoint_n_epoch: 1
num_epoch: 20
summary_every_n_steps: 1
infrequent_summary_every_n_steps: 0
last_step: 1
evaluation_window: 10
low_freq_cutoff: 0.67
high_freq_cutoff: 4.0
fps: 20.0
model_type: HRNet_release
dataloader:
  __class_name__: HeartRateNetDataloader
  image_info:
    num_channels: 3
    image_height: 72
    image_width: 72
    data_format: channels_first
  dataset_info:
    tfrecords_directory_path: /workspace/tao-experiments/heartratenet/data
    tfrecord_folder_name: ''
    tfrecord_train_file_name: train.tfrecord
    tfrecord_test_file_name: test.tfrecord
    tfrecord_validation_file_name: validation.tfrecord
  model_info:
    model_type: HRNet_release
model:
  __class_name__: HeartRateNet
  model_parameters:
    input_size: 72
    data_format: channels_first
    conv_dropout_rate: 0.0
    fully_connected_dropout_rate: 0.0
    use_batch_norm: False
    model_type: HRNet_release
    frozen_blocks: 0 # Freeze up the `nth` layer. Must be in range of [0, 5).
    pretrained_model: /workspace/tao-experiments/heartratenet/pretrain_models/heartratenet_vtrainable_v2.0/model.tlt
loss:
  __class_name__: HeartRateNetLoss
  loss_function_name: MSE
optimizer:
  __class_name__: AdadeltaOptimizer
  rho: 0.95
  epsilon: 1.0e-07
  learning_rate_schedule:
    __class_name__: ConstantLearningRateSchedule
    learning_rate: 1.0

For TAO training, please run in a dgpu machine or cloud machine.
After training, when run inference, user can run in dgpu machine, cloud machine and also Jetson devices.

I thought Jetson Nano would meet hardware requirements (except for 100 GB of SSD space. For which i have thought of a solution). If i switch to dgpu i would need to start all over again with setups. Plus, i would even need to setup TensorRT, CUDA and CUDNN or even worse, using cloud would be costly. Can’t i still be able to train on Jetson Nano if i sort out the SSD space issue?

Also, for training with custom dataset may i ask you for your advice and guidance please?

Thanks.

For TAO training, it is expected to run in x86-based CPU with dgpu or cloud gpu.
So, Jetson devices are not expected to run training.
For TAO training, it is running in docker. So, it is not needed to setup TensorRT/CUDA/Cudnn.

To get started for TAO, you can run with the notebook.

May i ask you to clarify, do i need to do anything outside of TAO Toolkit Quickstart to begin training with TAO?

Thanks.

Please check “Software Requirements” in TAO Toolkit Quick Start Guide - NVIDIA Docs.
One correction: python >=3.6.9 < 3.8.x

  • python >=3.6.9 < 3.8.x
  • docker-ce > 19.03.5
  • docker-API 1.40
  • nvidia-container-toolkit > 1.3.0-1
  • nvidia-container-runtime > 3.4.0-1
  • nvidia-docker2 > 2.5.0-1
  • nvidia-driver > 455+

Ubuntu: 20.04

May i ask you if it would work with Ubuntu versions other than Ubuntu 20.04? Since, it seems like i can’t make a dual boot with that version on Windows laptop.

Thanks.

Ubuntu 20.04 or Ubuntu18.04 are verified.

Thank you for the explanation.

Unfortunately, it seems like due to a hardware issue i can’t install Linux. So that, if you don’t mind may i ask you if i can follow TAO Toolkit Quickstart on WSL or WSL2 and if i would follow the quickstart guide on WSL or WSL2 would it work just as if i have installed the Ubuntu and working on that please?

Kindest regards.

P.S. I am asking especially about WSL/WSL2. Because, VMs doesn’t let me to allocate more than 128 MB for graphics.

User can use WSL for training. You can search and find some related topics in the forum.

Thank you so much for all the kind advice and guidance you have offered.

You can close this topic if you would like.

Kindest regards.

P.S. I am opening another topic for my questions on using WSL for training. Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.