Docker: Cannot connect to the Docker daemon

Hello,
I started using Linux a few months ago when I got a Jeton Nano for a Masters Course Project. I’ve had this issue with the jeston for about a week now, I’ve been searching for solutions, but none seem to work or address directly my issue. I’ll describe the problem and all the error messages i’ve obtained while trying to implement the solutions:

I tried to pull a docker using the following command:
sudo docker pull nvcr.io/nvidia/l4t-ml:r32.5.0-py3

It got installed properly, and no problema was shown, however just after I rebooted and connected to the jeston via SSH, I tried running the container using:
“sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-ml:r32.5.0-py3

“The following error message appeared: docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is he docker daemon running?.”

After used the following two commands I found on a blog:
“sudo systemctl unmask docker”
“systemctl start docker”

The latter caused this error message:
“Error getting authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.systemd1.ShuttingDown: Refusing activation, D-Bus is shutting down. (g-io-error-quark, 36)
Failed to start docker.service: Refusing activation, D-Bus is shutting down.
See system logs and ‘systemctl status docker.service’ for details.”

Then runned:
“systemctl status docker”

docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset:
Active: failed (Result: exit-code) since Fri 2021-04-30 20:05:46 -05; 13min
Docs: https://docs.docker.com
Main PID: 9805 (code=exited, status=1/FAILURE)

abr 30 20:05:46 linux systemd[1]: docker.service: Service hold-off time over, s
abr 30 20:05:46 linux systemd[1]: docker.service: Scheduled restart job, restar
abr 30 20:05:46 linux systemd[1]: Stopped Docker Application Container Engine.
abr 30 20:05:46 linux systemd[1]: docker.service: Start request repeated too qu
abr 30 20:05:46 linux systemd[1]: docker.service: Failed with result 'exit-code
abr 30 20:05:46 linux systemd[1]: Failed to start Docker Application Container

After searching more, I found I should restart Daemon, so I ran the following line:

“sudo dockerd”

I got the following error message:

"" INFO[2021-04-30T20:22:44.923076495-05:00] Starting up                          

INFO[2021-04-30T20:22:44.924775415-05:00] detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf
INFO[2021-04-30T20:22:44.925994592-05:00] parsed scheme: “unix” module=grpc
INFO[2021-04-30T20:22:44.926047614-05:00] scheme “unix” not registered, fallback to default scheme module=grpc
INFO[2021-04-30T20:22:44.926115427-05:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] } module=grpc
INFO[2021-04-30T20:22:44.926153239-05:00] ClientConn switching balancer to “pick_first” module=grpc
INFO[2021-04-30T20:22:44.928441019-05:00] parsed scheme: “unix” module=grpc
INFO[2021-04-30T20:22:44.928512218-05:00] scheme “unix” not registered, fallback to default scheme module=grpc
INFO[2021-04-30T20:22:44.928567427-05:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] } module=grpc
INFO[2021-04-30T20:22:44.928613104-05:00] ClientConn switching balancer to “pick_first” module=grpc
ERRO[2021-04-30T20:22:44.956576882-05:00] [graphdriver] prior storage driver overlay2 failed: lstat /var/lib/docker/overlay2/8d229224131676e6d0de1fce670d658a47467cbffeca0a401ee0cac0f1037634: structure needs cleaning
failed to start daemon: error initializing graphdriver: lstat /var/lib/docker/overlay2/8d229224131676e6d0de1fce670d658a47467cbffeca0a401ee0cac0f1037634: structure needs cleaning “”.

I would appreciate any help. And sorry if I make any mistake on the post, it’s my first time posting on any of these platforms. Thanks

Hi @aherrada, can you check the docker daemon logs with this command?

sudo journalctl -fu docker.service

Perhaps the error you got from running dockerd directory is a result of not starting it with overlay2 driver instead of graphdriver, alas I am not very familiar. I haven’t really seen your initial issue before, so if it persists, you may just want to completely un-install/re-install docker. You can find the related packages with:

dpkg -l | grep docker
dpkg -l | grep container

And then if you reinstall the nvidia-container-* packages via apt, hopefully that will get it back. Worse comes to worse, you may want to backup your work and reflash your SD card.

thanks so much, I’ll try it immediatly.

I thought the probem might be for an incorrect flashing of JetPack, in the previous flash I did, I tried uninsalling and instatlling it, it gave me the same error message when trying to delete remaining Docker files on the Jetson:
“Structure need cleaning”

Tha Journal just remains loading, doesn’t show any info besides this:
– Logs begin at Sun 2018-01-28 10:58:17 -05. –

I ran the two codes you told me:
jetson_uninorte@linux:~$ dpkg -l | grep docker
ii docker 1.5-1build1 arm64 System tray for KDE3/GNOME2 docklet applications
ii docker.io 19.03.6-0ubuntu1~18.04.2 arm64 Linux container runtime
ii nvidia-docker2 2.2.0-1 all nvidia-docker CLI wrapper
jetson_uninorte@linux:~$ dpkg -l | grep container
ii containerd 1.3.3-0ubuntu1~18.04.3 arm64 daemon to control runC
ii docker.io 19.03.6-0ubuntu1~18.04.2 arm64 Linux container runtime
ii libavformat57:arm64 7:3.4.8-0ubuntu0.2 arm64 FFmpeg library with (de)muxers for multimedia co ntainers - runtime files
ii libkf5wallet-bin 5.44.0-0ubuntu1 arm64 Secure and unified container for user passwords.
ii libkf5wallet-data 5.44.0-0ubuntu1 all Secure and unified container for user passwords.
ii libkf5wallet5:arm64 5.44.0-0ubuntu1 arm64 Secure and unified container for user passwords.
ii libkwalletbackend5-5:arm64 5.44.0-0ubuntu1 arm64 Secure and unified container for user passwords.
ii libmatroska6v5:arm64 1.4.8-1.1 arm64 extensible open standard audio/video container f ormat (shared library)
ii libnvidia-container-tools 0.9.0~beta.1 arm64 NVIDIA container runtime library (command-line t ools)
ii libnvidia-container0:arm64 0.9.0~beta.1 arm64 NVIDIA container runtime library
ii nvidia-container-csv-cuda 10.2.89-1 arm64 Jetpack CUDA CSV file
ii nvidia-container-csv-cudnn 8.0.0.180-1+cuda10.2 arm64 Jetpack CUDNN CSV file
ii nvidia-container-csv-tensorrt 7.1.3.0-1+cuda10.2 arm64 Jetpack TensorRT CSV file
ii nvidia-container-csv-visionworks 1.6.0.501 arm64 Jetpack VisionWorks CSV file
ii nvidia-container-runtime 3.1.0-1 arm64 NVIDIA container runtime
ii nvidia-container-toolkit 1.0.1-1 arm64 NVIDIA container runtime hook

If you are still getting the error after uninstalling/reinstalling docker packages, then I would recommend to re-flash your SD card so you can start fresh.

I’ve done about 5 times, is it possible that the SD card is damaged?

Hmm. If it is giving that error right after you flashed your SD card, then yes I would try using a different SD card instead.

Hi, today I went to retail store to change the micro SD for a new one, I mounted the latest JetPack zip on the SD card, and everything went alright. I was able to open a Container on the first boot. However, after I stopped the container and rebooting the system, it no longer functions. When trying to run the container it appears the following message:

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

Then tried to restart docker using: sudo dockerd

The following error appeared:

INFO[2021-05-03T18:18:37.289671742-05:00] Starting up
INFO[2021-05-03T18:18:46.947668042-05:00] parsed scheme: “unix” module=grpc
INFO[2021-05-03T18:18:46.948562181-05:00] scheme “unix” not registered, fallback to default scheme module=grpc
INFO[2021-05-03T18:18:46.948906461-05:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] } module=grpc
INFO[2021-05-03T18:18:46.949125686-05:00] ClientConn switching balancer to “pick_first” module=grpc
INFO[2021-05-03T18:18:46.960832521-05:00] parsed scheme: “unix” module=grpc
INFO[2021-05-03T18:18:46.961041017-05:00] scheme “unix” not registered, fallback to default scheme module=grpc
INFO[2021-05-03T18:18:46.961235606-05:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 }] } module=grpc
INFO[2021-05-03T18:18:46.961356651-05:00] ClientConn switching balancer to “pick_first” module=grpc
ERRO[2021-05-03T18:18:48.943280870-05:00] [graphdriver] prior storage driver overlay2 failed: lstat /var/lib/docker/overlay2/329692b2a9ecc59478a9a26155f452abb6b9225194ac0d5bef06e8bb27e5bb84: structure needs cleaning
failed to start daemon: error initializing graphdriver: lstat /var/lib/docker/overlay2/329692b2a9ecc59478a9a26155f452abb6b9225194ac0d5bef06e8bb27e5bb84: structure needs cleaning

Thanks for any help.

Are there any errors if you check dmesg, particularly about filesystem or when the docker daemon stopped? Strange it is still happening after you changed SD cards.

If you run sudo rm -r -f /var/lib/docker, does that allow your docker service to start again?

Can you try running l4t-base container to see if it causes same issue for you?

sudo docker run -it --rm --net=host --runtime nvidia nvcr.io/nvidia/l4t-base:r32.5.0