Hi NVIDIA team,
I’m starting to work with k3s and I’m trying to run a k3s cluster on a Jetson Xavier NX board with JetPack 4.6:
$ uname -srvmpio
Linux 4.9.253-tegra #1 SMP PREEMPT Mon Jul 26 12:19:28 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux
This is the example I’ve been trying to replicate: The first experience with k3s (lightweight Kubernetes). Deploy your first app! - DEV Community 👩💻👨💻
Every command seems to execute properly, but when I check the pods status, it shows CrashLoopBackOff:
$ k3s kubectl get ingress,svc,pods -n retail-project-dev
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/simple-rest-golang-ingress <none> * 192.168.0.17 80 21m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/simple-rest-golang-service ClusterIP 10.43.205.169 <none> 80/TCP 21m
NAME READY STATUS RESTARTS AGE
pod/simple-rest-golang-deployment-d96cb9d78-54k6v 0/1 CrashLoopBackOff 8 21m
pod/simple-rest-golang-deployment-d96cb9d78-224vf 0/1 CrashLoopBackOff 8 21m
I followed the same steps on my laptop:
$ uname -srvmpio
Linux 5.4.0-90-generic #101~18.04.1-Ubuntu SMP Fri Oct 22 09:25:04 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
And the pods get to running state without any issue:
$ k3s kubectl get ingress,svc,pods -n retail-project-dev
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/simple-rest-golang-ingress <none> * 192.168.0.27 80 2m51s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/simple-rest-golang-service ClusterIP 10.43.128.18 <none> 80/TCP 2m51s
NAME READY STATUS RESTARTS AGE
pod/simple-rest-golang-deployment-d96cb9d78-d8cn5 1/1 Running 0 2m51s
pod/simple-rest-golang-deployment-d96cb9d78-dmddt 1/1 Running 0 2m51s
I also used a script that is supposed to verify if the kernel configuration meet the requirements to run kubernetes. You can find the script here: moby/check-config.sh at master · moby/moby · GitHub
This is the result of running the script on the Jetson board:
$ ./check-config.sh
info: reading kernel config from /proc/config.gz ...
Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_NETFILTER_XT_MARK: enabled (as module)
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_POSIX_MQUEUE: enabled
- CONFIG_NF_NAT_IPV4: enabled (as module)
- CONFIG_NF_NAT_NEEDED: enabled
Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_SECCOMP_FILTER: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: enabled
(cgroup swap accounting is currently enabled)
- CONFIG_IOSCHED_CFQ: enabled
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: missing
- CONFIG_IP_NF_TARGET_REDIRECT: enabled (as module)
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_PROTO_TCP: enabled
- CONFIG_IP_VS_PROTO_UDP: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_SECURITY_SELINUX: missing
- CONFIG_SECURITY_APPARMOR: missing
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
- "overlay":
- CONFIG_VXLAN: enabled
- CONFIG_BRIDGE_VLAN_FILTERING: enabled
Optional (for encrypted networks):
- CONFIG_CRYPTO: enabled
- CONFIG_CRYPTO_AEAD: enabled
- CONFIG_CRYPTO_GCM: enabled
- CONFIG_CRYPTO_SEQIV: enabled
- CONFIG_CRYPTO_GHASH: enabled
- CONFIG_XFRM: enabled
- CONFIG_XFRM_USER: enabled
- CONFIG_XFRM_ALGO: enabled
- CONFIG_INET_ESP: enabled (as module)
- CONFIG_INET_XFRM_MODE_TRANSPORT: enabled
- "ipvlan":
- CONFIG_IPVLAN: enabled
- "macvlan":
- CONFIG_MACVLAN: enabled (as module)
- CONFIG_DUMMY: enabled
- "ftp,tftp client in container":
- CONFIG_NF_NAT_FTP: enabled (as module)
- CONFIG_NF_CONNTRACK_FTP: enabled (as module)
- CONFIG_NF_NAT_TFTP: enabled (as module)
- CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
- "aufs":
- CONFIG_AUFS_FS: missing
- "btrfs":
- CONFIG_BTRFS_FS: enabled (as module)
- CONFIG_BTRFS_FS_POSIX_ACL: enabled
- "devicemapper":
- CONFIG_BLK_DEV_DM: enabled
- CONFIG_DM_THIN_PROVISIONING: missing
- "overlay":
- CONFIG_OVERLAY_FS: enabled (as module)
- "zfs":
- /dev/zfs: missing
- zfs command: missing
- zpool command: missing
Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000
This shows that all the “Generally Necessary” options are enabled.
Is there any known issues with k3s and the Jetson boards or maybe related to JetPack 4.6 that may cause this behavior?
Thank you!