Error while running TensorFlow model on boot with systemd

Hello, I have a Python script to run the ssd_mobilenet_v2_coco TensorFlow model and it works fine from the terminal but it fails when starts from a systemd service.

This is the service file “red-neuronal.service” that I have created and pasted in /etc/systemd/system:

[Unit]
description=neural network

[Service]
Type=simple
ExecStart=/usr/bin/python3 /home/teraflop/Documentos/tensorrt_demos/trt_ssd_async.py --model ssd_mobilenet_v2_coco --usb --vid 0 --width 640 --height 480
user=TeraFlop

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl start red-neuronal.service

And with:

sudo systemctl status red-neuronal.service

I have the following error:

● red-neuronal.service
   Loaded: loaded (/etc/systemd/system/red-neuronal.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2020-02-17 22:24:54 -03; 1min 8s ago
   Process: 14117 ExecStart=/usr/bin/python3 /home/teraflop/Documentos/tensorrt_demos/trt_ssd_async.py --model ssd_mobilenet_v2_coco --usb --vid 0 --width 640 --height 480 (code=exited, status=1/FAILURE)
 Main PID: 14117 (code=exited, status=1/FAILURE)

feb 17 22:23:41 teraflop-ai-iot systemd[1]: Started red-neuronal.service.
feb 17 22:24:45 teraflop-ai-iot python3[14117]: 2020-02-17 22:24:45.023841: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
feb 17 22:24:53 teraflop-ai-iot python3[14117]: cannot open display: 
feb 17 22:24:54 teraflop-ai-iot systemd[1]: red-neuronal.service: Main process exited, code=exited, status=1/FAILURE
feb 17 22:24:54 teraflop-ai-iot systemd[1]: red-neuronal.service: Failed with result 'exit-code'.

Can someone help me? I was the whole day searching on internet and testing different configurations for the .service file and none has worked for me.
From already thank you very much.

Hi,

The issue looks from the display service:

feb 17 22:24:53 teraflop-ai-iot python3[14117]: cannot open display:

You can try to turn-off the display related code to see if works.(ex. imshow)
Or try to use local DISPLAY to see if helps.

export DISPLAY=:1

Thanks.

Thanks for for your response.

I did

export DISPLAY=:1

I proved this .service file

[Unit]
description=Neural Network

[Service]
Environment=DISPLAY=:1
Type=simple
ExecStart=/usr/bin/python3 /home/teraflop/Documentos/tensorrt_demos/trt_ssd_async.py --model ssd_mobilenet_v2_coco --usb --vid 0 --width 640 --height 480
user=TeraFlop

[Install]
WantedBy=multi-user.target

And I get the same error

● red-neuronal.service
   Loaded: loaded (/etc/systemd/system/red-neuronal.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2020-02-18 10:40:13 -03; 8s ago
  Process: 11347 ExecStart=/usr/bin/python3 /home/teraflop/Documentos/tensorrt_demos/trt_ssd_async.py --model ssd_mobilenet_v2_coco --usb --vid
 Main PID: 11347 (code=exited, status=1/FAILURE)

feb 18 10:40:01 teraflop-ai-iot systemd[1]: Started red-neuronal.service.
feb 18 10:40:08 teraflop-ai-iot python3[11347]: 2020-02-18 10:40:08.234636: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
feb 18 10:40:13 teraflop-ai-iot python3[11347]: cannot open display: :1
feb 18 10:40:13 teraflop-ai-iot systemd[1]: red-neuronal.service: Main process exited, code=exited, status=1/FAILURE
feb 18 10:40:13 teraflop-ai-iot systemd[1]: red-neuronal.service: Failed with result 'exit-code'.

I proved too

export DISPLAY=:0

And in the .service file

[Service] Environment=DISPLAY=:0

And I get

● red-neuronal.service
   Loaded: loaded (/etc/systemd/system/red-neuronal.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2020-02-18 10:46:03 -03; 3s ago
  Process: 11987 ExecStart=/usr/bin/python3 /home/teraflop/Documentos/tensorrt_demos/trt_ssd_async.py --model ssd_mobilenet_v2_coco --usb --vid
 Main PID: 11987 (code=exited, status=1/FAILURE)

feb 18 10:45:44 teraflop-ai-iot systemd[1]: Started red-neuronal.service.
feb 18 10:45:54 teraflop-ai-iot python3[11987]: 2020-02-18 10:45:54.310965: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
feb 18 10:46:02 teraflop-ai-iot python3[11987]: No protocol specified
feb 18 10:46:02 teraflop-ai-iot python3[11987]: cannot open display: :0
feb 18 10:46:03 teraflop-ai-iot systemd[1]: red-neuronal.service: Main process exited, code=exited, status=1/FAILURE
feb 18 10:46:03 teraflop-ai-iot systemd[1]: red-neuronal.service: Failed with result 'exit-code'.

Also the same error but this time “No protocol specified” appears.

Hi,

It looks like you will need to turn off the display related function.

We tested following script with systemd and it works correctly:

import tensorflow as tf
  
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
nvidia@xavier:/etc/systemd/system$ sudo systemctl status red-neuronal.service
● red-neuronal.service
   Loaded: loaded (/etc/systemd/system/red-neuronal.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

 三  04 17:11:59 xavier python3[31963]: 2020-03-04 17:11:59.995322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
 三  04 17:11:59 xavier python3[31963]: 2020-03-04 17:11:59.995350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
 三  04 17:11:59 xavier python3[31963]: 2020-03-04 17:11:59.995738: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
 三  04 17:11:59 xavier python3[31963]: 2020-03-04 17:11:59.995983: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:950] ARM64 does not support NUMA - returning NUMA node zero
 三  04 17:11:59 xavier python3[31963]: 2020-03-04 17:11:59.996221: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 236
 三  04 17:12:00 xavier python3[31963]: b'Hello, TensorFlow!'
 三  04 17:12:11 xavier systemd[1]: /etc/systemd/system/red-neuronal.service:2: Unknown lvalue 'description' in section 'Unit'
 三  04 17:12:11 xavier systemd[1]: /etc/systemd/system/red-neuronal.service:7: Unknown lvalue 'user' in section 'Service'
 三  04 17:14:25 xavier systemd[1]: /etc/systemd/system/red-neuronal.service:2: Unknown lvalue 'description' in section 'Unit'
 三  04 17:14:25 xavier systemd[1]: /etc/systemd/system/red-neuronal.service:7: Unknown lvalue 'user' in section 'Service'

Thanks.

In general, you can’t easily launch X apps through systemd. There are ways to launch graphical applications but all of them I know of will require you to either set up automatic login or disable X on startup entirely and then either set up a custom X unit or use an app that doesn’t require X, like gstreamer, which has egl and overlay sinks. If you need a graphical app on the boot, automatic login is certainly easier, but it shouldn’t be used in a kiosk type setting.