Remote script execution over ssh not working on Nano

Hi, I am trying to execute a python script remotely over ssh connection to the Jetson Nano but the Nano will not run the script. The script works on the Nano if I copy it to the Nano and then run it. But the script will not run if I pipe the script over ssh to the nano.

The same script will work using the same piping technique over ssh if the remote machine is not a Nano, ie not L4T.

This is my command:
#> cat myscript | ssh user@nano_IP python3

This is the log from ssh using the -vvv option

debug1: Authentication succeeded (password).
Authenticated to 10.xxxxxxxx ([10xxxxxxx]:22).
debug2: fd 4 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x08
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env SHELL
debug3: Ignored env SESSION_MANAGER
debug3: Ignored env WINDOWID
debug3: Ignored env QT_ACCESSIBILITY
debug3: Ignored env JETSON_TENSORRT
debug3: Ignored env COLORTERM
debug3: Ignored env XDG_CONFIG_DIRS
debug3: Ignored env XDG_SESSION_PATH
debug3: Ignored env XDG_MENU_PREFIX
debug3: Ignored env CLUTTER_BACKEND
debug3: Ignored env JETSON_CUDA_ARCH_BIN
debug3: Ignored env GTK2_MODULES
debug3: Ignored env LANGUAGE
debug3: Ignored env MANDATORY_PATH
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env JETSON_OPENCV_CUDA
debug3: Ignored env DESKTOP_SESSION
debug3: Ignored env SSH_AGENT_PID
debug3: Ignored env JETSON_VISIONWORKS
debug3: Ignored env GTK_MODULES
debug3: Ignored env XDG_SEAT
debug3: Ignored env PWD
debug3: Ignored env XDG_SESSION_DESKTOP
debug3: Ignored env LOGNAME
debug3: Ignored env QT_QPA_PLATFORMTHEME
debug3: Ignored env XDG_SESSION_TYPE
debug3: Ignored env PANEL_GDK_CORE_DEVICE_EVENTS
debug3: Ignored env GPG_AGENT_INFO
debug3: Ignored env JETSON_TYPE
debug3: Ignored env XAUTHORITY
debug3: Ignored env XDG_GREETER_DATA_DIR
debug3: Ignored env LD_PRELOAD
debug3: Ignored env GDM_LANG
debug3: Ignored env JETSON_BOARD
debug3: Ignored env HOME
debug1: Sending env LANG = en_AU.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env LS_COLORS
debug3: Ignored env XDG_CURRENT_DESKTOP
debug3: Ignored env VTE_VERSION
debug3: Ignored env XDG_SEAT_PATH
debug3: Ignored env JETSON_L4T
debug3: Ignored env GTK_CSD
debug3: Ignored env JETSON_SERIAL_NUMBER
debug3: Ignored env LESSCLOSE
debug3: Ignored env XDG_SESSION_CLASS
debug3: Ignored env TERM
debug3: Ignored env GTK_OVERLAY_SCROLLING
debug3: Ignored env DEFAULTS_PATH
debug3: Ignored env LESSOPEN
debug3: Ignored env LIBVIRT_DEFAULT_URI
debug3: Ignored env USER
debug3: Ignored env JETSON_CUDNN
debug3: Ignored env PAM_KWALLET5_LOGIN
debug3: Ignored env DISPLAY
debug3: Ignored env SHLVL
debug3: Ignored env JETSON_DESCRIPTION
debug3: Ignored env JETSON_OPENCV
debug3: Ignored env XDG_VTNR
debug3: Ignored env UBUNTU_MENUPROXY
debug3: Ignored env JETSON_L4T_RELEASE
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env LD_LIBRARY_PATH
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env XDG_DATA_DIRS
debug3: Ignored env BLAS
debug3: Ignored env PATH
debug3: Ignored env GDMSESSION
debug3: Ignored env JETSON_L4T_REVISION
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env ATLAS
debug3: Ignored env JETSON_CUDA
debug3: Ignored env JETSON_JETPACK
debug3: Ignored env OLDPWD
debug3: Ignored env _
debug1: Sending command: python3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 6 [write])
debug2: channel 0: input open → drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug3: send packet: type 96
debug2: channel 0: input drain → closed
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open → drain
debug2: channel 0: obuf empty
debug2: channel 0: chan_shutdown_write (i3 o1 sock -1 wfd 5 efd 6 [write])
debug2: channel 0: output drain → closed
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-signal reply 0
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
#0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/6 sock -1 cc -1)
debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 82964, received 2308 bytes, in 0.9 seconds
Bytes per second: sent 91588.9, received 2547.9
debug1: Exit status -1

It says the connection was successful, the file was transferred, but it will not execute. Here are the same debug lines from a non-Nano Ubuntu X86 machine running the same command, truncated to just the last few lines that differ from the Nano log

debug3: Ignored env ATLAS
debug3: Ignored env JETSON_CUDA
debug3: Ignored env JETSON_JETPACK
debug3: Ignored env OLDPWD
debug3: Ignored env _
debug1: Sending command: python3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 6 [write])
debug2: channel 0: input open → drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug3: send packet: type 96
debug2: channel 0: input drain → closed
debug2: client_check_window_change: changed
debug2: client_check_window_change: changed
debug2: client_check_window_change: changed
debug2: client_check_window_change: changed
debug2: client_check_window_change: changed
debug2: client_check_window_change: changed

I am using ssh-keygen -t rsa -b 4096 and ssh-copy-id -i to host, although it still asks for a password.

ssh auth log as follows:

tail /var/log/auth.log on Nano;

ct 16 21:48:50 x sshd[24761]: Accepted password for x from 10.1.1.250 port 46928 ssh2
Oct 16 21:48:50 x sshd[24761]: pam_unix(sshd:session): session opened for user x by (uid=0)
Oct 16 21:48:50 x systemd-logind[3867]: New session 262 of user x.
Oct 16 21:48:50 x sshd[25136]: Received disconnect from 10.1.1.250 port 46928:11: disconnected by user
Oct 16 21:48:50 x sshd[25136]: Disconnected from user x 10.1.1.250 port 46928
Oct 16 21:48:50 x sshd[24761]: pam_unix(sshd:session): session closed for user x
Oct 16 21:48:50 x systemd-logind[3867]: Removed session 262.

I also use ssh keys for working on Jetsons. Do you have a simple Python script which can be posted here for other people to test?

Also, are you expecting to display any remote graphics content? Is the GPU used anywhere in the Python script/

Thank you Linuxdev, see simple python script below, if I un-comment any of the imports the script does not run, but it will run with other imports like os, time, etc etc.

It looks like any imports with gpu dependencies causes it to fail.

#!/usr/bin

test imports

import os
import time
#import cv2
#import torch

print((“I am running”))

Are you using Python 2 or Python 3? Might be some differences between versions (due to different releases perhaps being in different directories), but basically the environment variables of a regular user are more set up than for when the system runs from the init process. You likely must set the PYTHONPATH. Here is a good URL on the topic:
https://bic-berkeley.github.io/psych-214-fall-2016/using_pythonpath.html

When a particular user wants to pass the environment variable to everything running after a certain point in code, then the user would “export” the variable. Example:

export PYTHONPATH=/some/where/with/modules
...python command runs here and inherits PYTHONPATH...

If you instead want this to be bound only to the current program and to not get its fingers into everything which follows (which is probably how you need this set up due to being run by the system), then here is an example of executing that code just for this one program by combining the variable and the execution program in a single line:

PYTHONPATH=/some/where/with/modules /my/program/to/runthis

(where “/my/program/to/runthis” is the program “runthis” as a full path, and PYTHONPATH names where to import files from)

Thanks linuxdev, much appreciated. Still not able to make it work.

I tried a new test script to see what user ssh thinks it is logged in to, then I use python to clear and then append sys.path, and I added every path I could find just in case, and then I check what it thinks the sys.path is set to, which is all correct, but still it does not load cv2, or numpy, or torch, etc. that exist in “/usr/local/lib/python3.6/dist-packages”

This is my new test script based on your answer above:

#!/usr/bin/python3

import sys
import os
import getpass

print(getpass.getuser())

sys.path.init()
sys.path.append("/usr/lib/python3.6/dist-packages")
sys.path.append("/usr/lib/python3/dist-packages")
sys.path.append("/usr/local/lib/python3.6/dist-packages")

for p in sys.path:
print(p)

try:
user_paths = os.environ[‘PYTHONPATH’].split(os.pathsep)
print(user_paths)
except KeyError:
user_paths =
print(" PYTHONPATH is not defined")

#import cv2

This is the output when run over ssh connection to user tx1
command i am using is

cat myscript | ssh tx1#10.x.x.x python3

output via ssh terminal session is :

tx1
/usr/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
/usr/local/lib/python3.6/dist-packages
PYTHONPATH is not defined

However, if I un-comment import cv2, it fails to run and there is no output.

If I create a PYTHONPATH on the remote device, it does not show up via this script, but I can print the PYTHONPATH on the remote machine. Maybe that is a clue ?

Any assistance appreciated

Remote login versus local login can change what the default environment is. Very definitely boot scripts have a different (much more restricted) environment than does a user login (regardless of whether it is via ssh or local). Since the script is failing to find PYTHONPATH (the log shows “PYTHONPATH is not defined”) I assume (perhaps incorrectly) that the path is being lost (or perhaps denied) due to some detail of how and when the script is run.

There would be a lot of output from the “env” command printing every single environment variable, and much of it would be useless or irrelevant, but if you were to have your script print the output of the command “env” just prior to the line which fails, and have it print once when it works (e.g., local login), and once when it fails, then that output could be compared. Keep in mind that there might be other “defaults” if PYTHONPATH is missing, and so figuring out where it is wrong won’t necessarily be straightforward.

To emphasize, environment variables depend on the shell, and the shell and the shell’s setup depend on who and when login occurs during the boot or later process.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.