Docker on the TX2

Autonomous Machines Jetson & Embedded Systems Jetson TX2
#1

Does anyone have any information about installing Docker on the TX2. When I run “sudo apt install docker.io” I get this error.

Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "start" failed.
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2017-03-20 02:15:34 UTC; 12ms ago
     Docs: https://docs.docker.com
  Process: 17303 ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS (code=exited, status=1/FAILURE)
 Main PID: 17303 (code=exited, status=1/FAILURE)

Mar 20 02:15:33 tegra-ubuntu systemd[1]: Starting Docker Application Container Engine...
Mar 20 02:15:33 tegra-ubuntu dockerd[17303]: time="2017-03-20T02:15:33.521844000Z" level=info msg="libcontainerd: new containerd process, pid: 17310"
Mar 20 02:15:34 tegra-ubuntu dockerd[17303]: time="2017-03-20T02:15:34.527480000Z" level=warning msg="unable to modify root key limit, number of containers could be limitied by this quota:...or directory"
Mar 20 02:15:34 tegra-ubuntu dockerd[17303]: time="2017-03-20T02:15:34.554898000Z" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new ...port loaded."
Mar 20 02:15:34 tegra-ubuntu dockerd[17303]: time="2017-03-20T02:15:34.558660000Z" level=error msg="There are no more loopback devices available."
Mar 20 02:15:34 tegra-ubuntu dockerd[17303]: time="2017-03-20T02:15:34.559010000Z" level=fatal msg="Error starting daemon: error initializing graphdriver: loopback attach failed"
Mar 20 02:15:34 tegra-ubuntu systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Mar 20 02:15:34 tegra-ubuntu systemd[1]: Failed to start Docker Application Container Engine.
Mar 20 02:15:34 tegra-ubuntu systemd[1]: docker.service: Unit entered failed state.
Mar 20 02:15:34 tegra-ubuntu systemd[1]: docker.service: Failed with result 'exit-code'.
Hint: Some lines were ellipsized, use -l to show in full.
dpkg: error processing package docker.io (--configure):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for systemd (229-4ubuntu16) ...
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

Any idea on how to fix this?

#2

Hi,

Thanks for your question.
This issue may caused by the missing file ‘/proc/sys/kernel/keys/root_maxkeys’.
Have you installed docker on aarch64 platform before?

We will keep investigating this issue and update to you later.
Thanks.

INFO[0000] libcontainerd: new containerd process, pid: 8143 
WARN[0000] containerd: low RLIMIT_NOFILE changing to max  current=1024 max=65536
WARN[0001] unable to modify root key limit, number of containers could be limitied by this quota: open /proc/sys/kernel/keys/root_maxkeys: no such file or directory 
ERRO[0001] There are no more loopback devices available. 
ERRO[0001] [graphdriver] prior storage driver "devicemapper" failed: loopback attach failed 
FATA[0001] Error starting daemon: error initializing graphdriver: loopback attach failed
#3

Our goal is to use the same docker containers as our project on the Raspberry Pi 3. In theory it should work. They are both ARMv8 64 bit. Dieter Reuter(https://twitter.com/Quintus23M) one of the creators of Hypriot OS (http://blog.hypriot.com) tweeted that he would get it running if he had an Jetson TX2 to work on. He has already run docker on other aarch64 systems.

It looks like the kernel is missing some options that docker needs. Docker provides a utility called check-config.sh that look for kernel config issues. Here is the output.

info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: missing
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: missing
- CONFIG_KEYS: missing
- CONFIG_VETH: missing
- CONFIG_BRIDGE: missing
- CONFIG_BRIDGE_NETFILTER: missing
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: missing
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: missing
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: missing
- CONFIG_DEVPTS_MULTIPLE_INSTANCES: enabled

Optional Features:
- CONFIG_USER_NS: missing
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: missing
- CONFIG_MEMCG_SWAP: missing
- CONFIG_MEMCG_SWAP_ENABLED: missing
- CONFIG_MEMCG_KMEM: missing
- CONFIG_BLK_CGROUP: missing
- CONFIG_BLK_DEV_THROTTLING: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: missing
- CONFIG_CGROUP_HUGETLB: missing
- CONFIG_NET_CLS_CGROUP: missing
- CONFIG_CGROUP_NET_PRIO: missing
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_VS: missing
- CONFIG_IP_VS_NFCT: missing
- CONFIG_IP_VS_RR: missing
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: missing
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: missing
      - CONFIG_CRYPTO_SEQIV: missing
      - CONFIG_CRYPTO_GHASH: missing
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled
      - CONFIG_XFRM_ALGO: enabled
      - CONFIG_INET_ESP: enabled
      - CONFIG_INET_XFRM_MODE_TRANSPORT: enabled
  - "ipvlan":
    - CONFIG_IPVLAN: missing
  - "macvlan":
    - CONFIG_MACVLAN: missing
    - CONFIG_DUMMY: enabled
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: missing
    - CONFIG_BTRFS_FS_POSIX_ACL: missing
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled
    - CONFIG_DM_THIN_PROVISIONING: missing
  - "overlay":
    - CONFIG_OVERLAY_FS: missing
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
cat: /proc/sys/kernel/keys/root_maxkeys: No such file or directory
./check-config.sh: line 347: [: -le: unary operator expected
cat: /proc/sys/kernel/keys/root_maxkeys: No such file or directory
- /proc/sys/kernel/keys/root_maxkeys:
#4

Yes I have installed docker.io on aarch64, e.g. the TX1, Odroid C2.

With the TX1, you simply install it with apt-get install docker.io, with TX2. No such luck. Other support is missing e.g. no loopback devices. It is also preferable to use aufs (IMHO). So supporting that would be helpful (Ubuntu 16.04 on Odroid, RPI, etc… includes AUFS).

Thanks.

#5

I am trying to recompile the kernel to get it installed. No joy so far. I get passed one error and just hit an other. It’s good to know it’s possible so I’ll keep trying.

#6

OK. Got it.

Here is the config I used.

https://github.com/frankjoshua/buildJetsonTX2Kernel/blob/master/docker_config/config

I rebuilt the kernel using instructions from https://github.com/jetsonhacks/buildJetsonTX2Kernel

Run getKernelSources.sh then close out the kernel editor gui after it starts. Then copy the file ./docker_conf/config to /usr/src/kernel/kernel-4.4/.config

Then run makeKernel.sh and copyImage.sh.

After you reboot “uname -r” should display 4.4.15-docker

Finally
sudo apt install docker.io

No aufs support yet but so far everything is working. I am running docker containers the were originally created for the Raspberry Pi 3 and they work great so far.

#7

when i run:

% sudo apt install docker.io

i get:

Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Unable to locate package docker.io
…

thoughts?

i’d love to get get docker on my TX2 given the above information but i am presently blocked upstream.

thx,

  • james
#8

James,

Here is my /etc/apt/sources.list

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.

deb http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial main restricted universe multiverse

## Major bug fix updates produced after the final release of the
## distribution.
deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates main restricted universe multiverse

## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
# deb http://ports.ubuntu.com/ubuntu-ports/ xenial universe
# deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial universe
# deb http://ports.ubuntu.com/ubuntu-ports/ xenial-updates universe
# deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-updates universe

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted
# deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-backports main restricted

deb http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb-src http://ports.ubuntu.com/ubuntu-ports/ xenial-security main restricted universe multiverse
deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable

sudo apt update

Sorry I don’t remember all of the steps I took to get Docker working. I spent a week on it and most of what I did was probably not needed.

Please let me know if you get Docker working also. I want to do a clean install on my TX2 but I’m worried I won’t get it working a second time.

#9

hmmmm … some of the comments on this thread have been removed?

specifically the links to: https://github.com/jetsonhacks/buildJetsonTX2Kernel

odd

  • james
#10

Had the same issue today. Did not seem to be a full device issue.

#11

Hi,

Thanks for your question.
Not sure why previous comment is removed.

Here is the information:

Please re-compile rel-27.1 kernel with this config to enable required options:

Instruction for recompiling kernel is here:

Thanks.

#12

Thanks for this very helpful information. I was able to compile using the kernel config you gave but I did it on a host computer and cross-compiled. Compile completed successfully and I was able to flash and the image looks to be fine, but there is no WiFi.

Just wondering if the kernel that you loaded has WiFi enabled or not. Any ideas on if this is a kernel issue or if there is something else I might be missing?

#13

A comment on loopback…this is kernel config CONFIG_BLK_DEV_LOOP. It isn’t set by default on the TX2, but probably should be. This would be a change worth putting in to the next L4T release.

#14

Figured out the WiFi issue. I wasn’t recompiling on the host but was rather crosscompiling and moving the kernel over. When I did this I forgot to also move the modules over as well. Once I moved these over and rebooted, wifi was working again.

#15

We are able to get docker to work on the TX-2 now without issues but was wondering if anyone has had success getting GPU programs to work in a docker container on the TX-2.

I know there is an nvidia-docker project but it does not support the TX devices and probably won’t in the near future (see here for more info: https://github.com/NVIDIA/nvidia-docker/issues/214

Tried various ways to get something simple like deviceQuery to work but keep getting errors like:

cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL

I think the drivers work very differently on the TX-2 than they do on machines with an external GPU but don’t have a good feeling for what we might be missing to get this working.

#16

Please advise.Error as shown.

nvidia@tegra-ubuntu:~$ sudo apt install docker.io
Reading package lists… Done
Building dependency tree
Reading state information… Done
docker.io is already the newest version (1.12.6-0ubuntu1~16.04.1).
The following packages were automatically installed and are no longer required:
apt-clone archdetect-deb dmeventd dmraid dpkg-repack gir1.2-timezonemap-1.0 gir1.2-xkl-1.0 kpartx kpartx-boot libdebian-installer4
libdevmapper-event1.02.1 libdmraid1.0.0.rc16 liblockfile-bin liblockfile1 liblvm2app2.2 liblvm2cmd2.02 libparted-fs-resize0
libreadline5 lockfile-progs lvm2 os-prober pmount python3-icu python3-pam rdate ubiquity-casper ubiquity-ubuntu-artwork
Use ‘sudo apt autoremove’ to remove them.
0 upgraded, 0 newly installed, 0 to remove and 239 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up docker.io (1.12.6-0ubuntu1~16.04.1) …
addgroup: The group `docker’ already exists as a system group. Exiting.
Job for docker.service failed because the control process exited with error code. See “systemctl status docker.service” and “journalctl -xe” for details.
invoke-rc.d: initscript docker, action “start” failed.
dpkg: error processing package docker.io (–configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

#17

I don’t know docker setup, but what is the output of this:

sudo systemctl status docker.service

(it looks like starting docker may require a service to be running)

#18

Was able to get GPU programs running in Docker containers on the TX-2. Should work for other Tegra devices as well, but right now we only have the TX-2 to test with.

Instructions are at this GitHub repo, https://github.com/Technica-Corporation/Tegra-Docker

#19

@kgdad

  Thank you very much i will try.
#20

I followed the instructions in the github link but it failed on getting the GPU to work in the docker, cudaGetDeviceCount fails and returns 35. Any ideas?