subprocess.CalledProcessError: Command '['nvidia-smi', '-L']' returned non-zero exit status 255

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc)

RTX2060

• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc)

EmotionNet

• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here)

~/.tao_mounts.json wasn’t found. Falling back to obtain mount points and docker configs from ~/.tlt_mounts.json.
Please note that this will be deprecated going forward.
Configuration of the TAO Toolkit Instance
dockers: [‘nvidia/tao/tao-toolkit-tf’, ‘nvidia/tao/tao-toolkit-pyt’, ‘nvidia/tao/tao-toolkit-lm’]
format_version: 2.0
toolkit_version: 3.22.02
published_date: 02/28/2022

• Training spec file(If have, please share here)

class_name: EmotionNetTrainer
checkpoint_dir: null
random_seed: 42
log_every_n_secs: 10
checkpoint_n_epoch: 1
num_epoch: 50
infrequent_summary_every_n_steps: 0
use_landmarks_input: True
class_list: [‘neutral’,
‘happy’,
‘surprise’,
‘contempt’,
‘disgust’,
‘angry’]
dataloader:
class_name: EmotionNetDataloader
batch_size: 64
face_scale_factor: 1.3
num_keypoints: 68
prefetch_num: 3
image_info:
image_frame:
channel: 1
height: 480
width: 640
image_face:
channel: 1
height: 224
width: 224
dataset_info:
root_path: null
image_extension: png
tfrecords_directory_path:
- /workspace/tao-experiments/emotionnet/postData
tfrecords_set_id:
- ckplus
ground_truth_folder_name:
- Ground_Truth_DataFactory
tfrecord_folder_name:
- TfRecords_combined
train_file_name: train.tfrecords
validate_file_name: validate.tfrecords
test_file_name: test.tfrecords
kpiset_info:
kpi_root_path: null
kpi_tfrecords_directory_path:
- /workspace/tao-experiments/emotionnet/postData
tfrecords_set_id_kpi:
- ckplus
ground_truth_folder_name_kpi:
- Ground_Truth_DataFactory
tfrecord_folder_name_kpi:
- TfRecords_combined
kpi_file_name: test.tfrecords
model:
class_name: EmotionNetModel
model_parameters:
use_batch_norm: True
data_format: channels_first
regularization_type: l2
regularization_factor: 0.002
bias_regularizer: null
use_landmarks_input: True
activation_type: ‘relu’
dropout_rate: 0.3
num_class: 6
pretrained_model_path: /workspace/tao-experiments/emotionnet/pretrain_models/emotionnet_vtrainable_v1.0/model.tlt
frozen_blocks: 2
loss:
class_name: EmotionNetLoss
loss_function_name: CE
class_weights_dict: None
optimizer:
class_name: AdamOptimizer
beta1: 0.9
beta2: 0.999
epsilon: 1.0e-08
learning_rate_schedule:
class_name: SoftstartAnnealingLearningRateSchedule
soft_start: 0.2
annealing: 0.8
base_learning_rate: 0.0002
min_learning_rate: 2.0e-07
last_step: 953801
evaluator:
class_name: EmotionNetEvaluator
dataloader:
class_name: EmotionNetDataloader
batch_size: 1
face_scale_factor: 1.3
num_keypoints: 68
prefetch_num: 3
image_info:
image_frame:
channel: 1
height: 480
width: 640
image_face:
channel: 1
height: 224
width: 224
dataset_info:
root_path: null
image_extension: png
tfrecords_directory_path:
- /workspace/tao-experiments/emotionnet/postData
tfrecords_set_id:
- ckplus
ground_truth_folder_name:
- Ground_Truth_DataFactory
tfrecord_folder_name:
- TfRecords_combined
train_file_name: train.tfrecords
validate_file_name: validate.tfrecords
test_file_name: test.tfrecords
kpiset_info:
kpi_root_path: null
kpi_tfrecords_directory_path:
- /workspace/tao-experiments/emotionnet/postData
tfrecords_set_id_kpi:
- ckplus
ground_truth_folder_name_kpi:
- Ground_Truth_DataFactory
tfrecord_folder_name_kpi:
- TfRecords_combined
kpi_file_name: test.tfrecords
loss:
class_name: EmotionNetLoss
loss_function_name: CE

• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

put all stuff in the same folder and run bash:
export USER_EXPERIMENT_DIR=./
tao emotionnet inference -e $USER_EXPERIMENT_DIR/emotionnet_tlt_pretrain.yaml
-i $USER_EXPERIMENT_DIR/inferInputs/001.json
-m $USER_EXPERIMENT_DIR/model.etlt
-r $USER_EXPERIMENT_DIR/inferOutputs
-o $USER_EXPERIMENT_DIR/inferOutputs
-k nvidia_tlt

Can you share the full log ? And did you ever follow the jupyter notebook?

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.