Kube proxy in iptables mode fails

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
,
1

We are deploying Kubernetes clusters on Clara AGX (JetPack 5.1) as part of our edge-cloud architecture federated with Karmada. On worker nodes, Kube proxy in iptables mode fails. Kubeproxy can only write partial rules. It can not implement all the rules required by the Kubernetes to enable communication between pods. As a result when trying to install Karmada agent, it can not communicate to the API server of the local cluster hence local cluster can not join remote Karmada control plane. Installing iptables/xtables packages did not help. It seems the kernel does not provide the required netfilter extension.

Someone familiar with Linux kernel configuration on Clara (L4T kernels), especially netfilter/xtables modules (xt_nfacct, nf_conntrack, nf_nat, etc.) might be able to help us.

Cluster setup:
ubuntu@clara-old:~$ uname -r
5.10.65-tegra

ubuntu@clara-old:~$ kubectl version
Client Version: v1.31.13
Kustomize Version: v5.4.2
The connection to the server localhost:8080 was refused - did you specify the right host or port?

ubuntu@clara-old:~$ kubelet --version
Kubernetes v1.31.13

ubuntu@clara-old:~$ containerd --version
containerd containerd.io 1.7.27 05044ec0a9a75232cad458027ca83437aae3f4da

ubuntu@clara-old:~$ ctr version
Client:
Version: 1.7.27
Revision: 05044ec0a9a75232cad458027ca83437aae3f4da
Go version: go1.23.7

ctr: failed to dial “/run/containerd/containerd.sock”: connection error: desc = “transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied”

Kubeproxy has written iptables rules partially.

Screenshot 2025-11-12 235530
Screenshot 2025-11-12 2355301371×208 9.05 KB

I appreciate your cordial support. Thanks!

4

Hi,

Do you need iptables-nft to enable the proxy for your use case?

Please note that iptables-nft is only supported in the rel-38 branch.
We have tried to enable it in the other branch (ex., rel-36) with some extra kernel configuration but no luck.
Please find more details in the topic below:

A workaround to enable Kubernetes without iptables-nft is to servicelb.
Please check more details for the WAR below:

Thanks.

5

ubuntu@clara-old:~$ sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
[sudo] password for ubuntu:
INFO[0000] Auto-detected mode as “nvml”
INFO[0000] Selecting /dev/nvidia0 as /dev/nvidia0
INFO[0000] Selecting /dev/dri/card0 as /dev/dri/card0
WARN[0000] Could not locate /dev/dri/controlD64: pattern /dev/dri/controlD64 not found
INFO[0000] Selecting /dev/dri/renderD128 as /dev/dri/renderD128
WARN[0000] Failed to evaluate symlink /dev/dri/by-path/pci-0000:09:00.0-card; ignoring
WARN[0000] Failed to evaluate symlink /dev/dri/by-path/pci-0000:09:00.0-render; ignoring
INFO[0000] Using driver version 570.133.07
INFO[0000] Selecting /dev/nvidia-modeset as /dev/nvidia-modeset
INFO[0000] Selecting /dev/nvidia-uvm-tools as /dev/nvidia-uvm-tools
INFO[0000] Selecting /dev/nvidia-uvm as /dev/nvidia-uvm
INFO[0000] Selecting /dev/nvidiactl as /dev/nvidiactl
WARN[0000] Could not locate libnvidia-egl-gbm.so: 64-bit library libnvidia-egl-gbm.so not found
INFO[0000] Selecting /usr/share/glvnd/egl_vendor.d/10_nvidia.json as /usr/share/glvnd/egl_vendor.d/10_nvidia.json
INFO[0000] Selecting /usr/share/vulkan/icd.d/nvidia_icd.json as /usr/share/vulkan/icd.d/nvidia_icd.json
INFO[0000] Selecting /usr/share/vulkan/implicit_layer.d/nvidia_layers.json as /usr/share/vulkan/implicit_layer.d/nvidia_layers.json
INFO[0000] Selecting /usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json as /usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json
INFO[0000] Selecting /usr/share/egl/egl_external_platform.d/10_nvidia_wayland.json as /usr/share/egl/egl_external_platform.d/10_nvidia_wayland.json
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libGLESv1_CM_nvidia.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libGLX_nvidia.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libcuda.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libcuda.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libcudadebugger.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libcudadebugger.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvcuvid.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvcuvid.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-allocator.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-cfg.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-eglcore.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-encode.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-encode.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-fbc.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-glcore.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-glsi.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-glvkspirv.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-gpucomp.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-gpucomp.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-ml.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-ml.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-ngx.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-opencl.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-opticalflow.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-ptxjitcompiler.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-rtcore.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvidia-tls.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvidia-tls.so.570.133.07
INFO[0000] Selecting /usr/lib/aarch64-linux-gnu/libnvoptix.so.570.133.07 as /usr/lib/aarch64-linux-gnu/libnvoptix.so.570.133.07
WARN[0000] Could not locate /nvidia-persistenced/socket: pattern /nvidia-persistenced/socket not found
WARN[0000] Could not locate /nvidia-fabricmanager/socket: pattern /nvidia-fabricmanager/socket not found
WARN[0000] Could not locate /tmp/nvidia-mps: pattern /tmp/nvidia-mps not found
INFO[0000] Selecting /lib/firmware/nvidia/570.133.07/gsp_ga10x.bin as /lib/firmware/nvidia/570.133.07/gsp_ga10x.bin
INFO[0000] Selecting /lib/firmware/nvidia/570.133.07/gsp_tu10x.bin as /lib/firmware/nvidia/570.133.07/gsp_tu10x.bin
INFO[0000] Selecting /usr/bin/nvidia-smi as /usr/bin/nvidia-smi
INFO[0000] Selecting /usr/bin/nvidia-debugdump as /usr/bin/nvidia-debugdump
INFO[0000] Selecting /usr/bin/nvidia-persistenced as /usr/bin/nvidia-persistenced
INFO[0000] Selecting /usr/bin/nvidia-cuda-mps-control as /usr/bin/nvidia-cuda-mps-control
INFO[0000] Selecting /usr/bin/nvidia-cuda-mps-server as /usr/bin/nvidia-cuda-mps-server
INFO[0000] Generated CDI spec with version 0.5.0
ubuntu@clara-old:~$ sudo nvidia-ctk runtime configure --runtime=docker --cdi.enabled=true
Incorrect Usage: flag provided but not defined: -cdi.enabled

NAME:
NVIDIA Container Toolkit CLI runtime configure - Add a runtime to the specified container engine

USAGE:
NVIDIA Container Toolkit CLI runtime configure [command options] [arguments…]

OPTIONS:
–dry-run update the runtime configuration as required but don’t write changes to disk (default: false)
–runtime value the target runtime engine. One of [crio, docker] (default: “docker”)
–config value path to the config file for the target runtime
–nvidia-runtime-name value specify the name of the NVIDIA runtime that will be added (default: “nvidia”)
–runtime-path value specify the path to the NVIDIA runtime executable (default: “nvidia-container-runtime”)
–set-as-default set the specified runtime as the default runtime (default: false)
–help, -h show help (default: false)

ERRO[0000] flag provided but not defined: -cdi.enabled
ubuntu@clara-old:~$ sudo systemctl restart docker^C
ubuntu@clara-old:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ubuntu@clara-old:~$ nvidia-ctk cdi list INFO[0000] Found 4 CDI devices nvidia.com/gpu=0 nvidia.com/gpu=all nvidia.com/pva=0 nvidia.com/pva=all
No help topic for ‘list’
ubuntu@clara-old:~$ nvidia-ctk cdi list
No help topic for ‘list’
ubuntu@clara-old:~$ nvidia-ctk cdi
NAME:
NVIDIA Container Toolkit CLI cdi - Provide tools for interacting with Container Device Interface specifications

USAGE:
NVIDIA Container Toolkit CLI cdi command [command options] [arguments…]

COMMANDS:
generate Generate CDI specifications for use with CDI-enabled runtimes
transform Apply a transform to a CDI specification
help, h Shows a list of commands or help for one command

OPTIONS:
–help, -h show help (default: false)

6

Hi,

We test the above command on the JetPack 6.2.1 environment, and it can work as expected.

$ nvidia-ctk cdi list
INFO[0000] Found 4 CDI devices                          
nvidia.com/gpu=0
nvidia.com/gpu=all
nvidia.com/pva=0
nvidia.com/pva=all

In our environment, the CDI version is 0.8.0.
Could you try to upgrade your environment and try it again?

Thanks.

8

Before upgrading the Jetpack, I was trying to see if we can replace Kubeproxy with Cilium eBPF. Unfortunately, I am gettting “unable to initialize kube-proxy replacement options: socketlb enabled, but kernel does not support attaching bpf programs to cgroups: create cgroup program” error from the Cilium agent running on the Clara AGX device. I am giving this information here for future reference. Here is the full log of the Cilium agent container.

level=info msg=" || || _ _" subsys=daemon
level=info msg="| | | | | | | |" subsys=daemon
level=info msg="||||||||_|" subsys=daemon
level=info msg=“Cilium 1.15.0 2db45c46 2024-01-31T18:58:02+01:00 go version go1.21.6 linux/arm64” subsys=daemon
level=info msg=“clang (10.0.0) and kernel (5.10.65) versions: OK!” subsys=linux-datapath
level=warning msg=“BPF system config check: NOT OK.” error=“CONFIG_NET_CLS_BPF kernel parameter or module is required (needed for: Essential eBPF infrastructure)” subsys=linux-datapath
level=info msg=“Detected mounted BPF filesystem at /sys/fs/bpf” subsys=bpf
level=info msg=“Mounted cgroupv2 filesystem at /run/cilium/cgroupv2” subsys=cgroups
level=info msg=“Parsing base label prefixes from default label list” subsys=labels-filter
level=info msg=“Parsing additional label prefixes from user inputs: ” subsys=labels-filter
level=info msg=“Final label prefixes to be used for identity evaluation:” subsys=labels-filter
level=info msg=" - reserved:.*" subsys=labels-filter
level=info msg=" - :io\.kubernetes\.pod\.namespace" subsys=labels-filter
level=info msg=" - :io\.cilium\.k8s\.namespace\.labels" subsys=labels-filter
level=info msg=" - :app\.kubernetes\.io" subsys=labels-filter
level=info msg=" - !:io\.kubernetes" subsys=labels-filter
level=info msg=" - !:kubernetes\.io" subsys=labels-filter
level=info msg=" - !:statefulset\.kubernetes\.io/pod-name" subsys=labels-filter
level=info msg=" - !:apps\.kubernetes\.io/pod-index" subsys=labels-filter
level=info msg=" - !:batch\.kubernetes\.io/job-completion-index" subsys=labels-filter
level=info msg=" - !:.beta\.kubernetes\.io" subsys=labels-filter
level=info msg=" - !:k8s\.io" subsys=labels-filter
level=info msg=" - !:pod-template-generation" subsys=labels-filter
level=info msg=" - !:pod-template-hash" subsys=labels-filter
level=info msg=" - !:controller-revision-hash" subsys=labels-filter
level=info msg=" - !:annotation." subsys=labels-filter
level=info msg=" - !:etcd_node" subsys=labels-filter
level=info msg=Invoked duration=2.955209ms function=“pprof.glob..func1 (pkg/pprof/cell.go:50)” subsys=hive
level=info msg=Invoked duration=“301.838µs” function=“gops.registerGopsHooks (pkg/gops/cell.go:38)” subsys=hive
level=info msg=Invoked duration=3.691276ms function=“metrics.glob..func1 (pkg/metrics/cell.go:13)” subsys=hive
level=info msg=Invoked duration=“124.614µs” function=“metricsmap.RegisterCollector (pkg/maps/metricsmap/metricsmap.go:281)” subsys=hive
level=info msg=“Spire Delegate API Client is disabled as no socket path is configured” subsys=spire-delegate
level=info msg=“Mutual authentication handler is disabled as no port is configured” subsys=auth
level=info msg=Invoked duration=176.812394ms function=“cmd.configureAPIServer (cmd/cells.go:215)” subsys=hive
level=info msg=Invoked duration=“37.794µs” function=“cmd.unlockAfterAPIServer (cmd/deletion_queue.go:113)” subsys=hive
level=info msg=Invoked duration=“99.876µs” function=“controller.Init (pkg/controller/cell.go:67)” subsys=hive
level=info msg=Invoked duration=“355.696µs” function=“endpointcleanup.registerCleanup (pkg/endpointcleanup/cleanup.go:66)” subsys=hive
level=info msg=Invoked duration=“16.961µs” function=“cmd.glob..func3 (cmd/daemon_main.go:1613)” subsys=hive
level=info msg=Invoked duration=“163.528µs” function=“cmd.registerEndpointBPFProgWatchdog (cmd/watchdogs.go:57)” subsys=hive
level=info msg=Invoked duration=“119.654µs” function=“envoy.registerEnvoyVersionCheck (pkg/envoy/cell.go:132)” subsys=hive
level=info msg=Invoked duration=“146.343µs” function=“cmd.registerDeviceReloader (cmd/device-reloader.go:48)” subsys=hive
level=info msg=Invoked duration=“54.882µs” function=“utime.initUtimeSync (pkg/datapath/linux/utime/cell.go:31)” subsys=hive
level=info msg=Invoked duration=“240.747µs” function=“agentliveness.newAgentLivenessUpdater (pkg/datapath/agentliveness/agent_liveness.go:43)” subsys=hive
level=info msg=Invoked duration=“60.323µs” function=“statedb.RegisterTable[…] (pkg/statedb/db.go:121)” subsys=hive
level=info msg=Invoked duration=“137.991µs” function=“l2responder.NewL2ResponderReconciler (pkg/datapath/l2responder/l2responder.go:72)” subsys=hive
level=info msg=Invoked duration=“148.326µs” function=“garp.newGARPProcessor (pkg/datapath/garp/processor.go:27)” subsys=hive
level=info msg=Invoked duration=“20.225µs” function=“bigtcp.glob..func1 (pkg/datapath/linux/bigtcp/bigtcp.go:58)” subsys=hive
level=info msg=Invoked duration=“23.329µs” function=“linux.glob..func1 (pkg/datapath/linux/devices_controller.go:62)” subsys=hive
level=info msg=Invoked duration=“123.558µs” function=“ipcache.glob..func3 (pkg/datapath/ipcache/cell.go:25)” subsys=hive
level=info msg=Starting subsys=hive
level=info msg=“Started gops server” address=“127.0.0.1:9890” subsys=gops
level=info msg=“Start hook executed” duration=“995.566µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“7.456µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Establishing connection to apiserver” host=“https://192.168.1.46:6443” subsys=k8s-client
level=info msg=“Serving prometheus metrics on :9962” subsys=metrics
level=info msg=“Connected to apiserver” subsys=k8s-client
level=info msg=“Start hook executed” duration=84.778924ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“732.163µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“289.294µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=1.342527ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“303.758µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=1.02872ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“204.778µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“21.729µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Using autogenerated IPv4 allocation range” subsys=node v4Prefix=10.23.0.0/16
level=info msg=“no local ciliumnode found, will not restore cilium internal ips from k8s” subsys=daemon
level=info msg=“Start hook executed” duration=126.86818ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“109.669µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“142.535µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“46.786µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Devices changed” devices=“[eth0]” subsys=devices-controller
level=info msg=“Start hook executed” duration=9.041253ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Node addresses updated” device=eth0 node-addresses=“192.168.1.23 (eth0)” subsys=node-address
level=info msg=“Node addresses updated” device=l4tbr0 node-addresses=“192.168.55.1 (l4tbr0)” subsys=node-address
level=info msg=“Node addresses updated” device=br-1eec5cf2c3c4 node-addresses=“172.18.0.1 (br-1eec5cf2c3c4)” subsys=node-address
level=info msg=“Start hook executed” duration=“847.016µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=1.335774ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=1.669102ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=17.386761ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“29.409µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“33.378µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“15.296µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“8.736µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“5.152µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.784µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Restored 0 node IDs from the BPF map” subsys=linux-datapath
level=info msg=“Start hook executed” duration=“81.956µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“10.305µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.848µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“8.224µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.464µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“3.52µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“3.584µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.784µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.529µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.912µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“2.784µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“4.353µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“5.504µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“5.024µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“52.034µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“221.098µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“1.984µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Using discoveryv1.EndpointSlice” subsys=k8s
level=info msg=“Serving cilium node monitor v1.2 API at unix:///var/run/cilium/monitor1_2.sock” subsys=monitor-agent
level=info msg=“Start hook executed” duration=“741.187µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Generating CNI configuration file with mode none” subsys=cni-config
level=info msg=“Start hook executed” duration=“8.193µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“190.889µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“27.329µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“351.697µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Envoy: Starting access log server listening on /var/run/cilium/envoy/sockets/access_log.sock” subsys=envoy-manager
level=info msg=“Start hook executed” duration=“91.748µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“514.52µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“33.889µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Envoy: Starting xDS gRPC server listening on /var/run/cilium/envoy/sockets/xds.sock” subsys=envoy-manager
level=info msg=“Start hook executed” duration=2.632315ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Datapath signal listener running” subsys=signal
level=info msg=“Start hook executed” duration=4.884163ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“55.65µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“144.55µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“20.321µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=1.408738ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“147.719µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“319.407µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Start hook executed” duration=“4.161µs” function=cell.augmentedHook.Start subsys=hive
level=info msg=“Inheriting MTU from external network interface” device=eth0 ipAddr=192.168.1.23 mtu=1500 subsys=mtu
level=info msg=“Start hook executed” duration=1.067954ms function=cell.augmentedHook.Start subsys=hive
level=info msg=“Auto-enabling “enable-node-port”, “enable-external-ips”, “bpf-lb-sock”, “enable-host-port”, “enable-session-affinity” features” subsys=daemon
level=error msg=“unable to initialize kube-proxy replacement options” error=“socketlb enabled, but kernel does not support attaching bpf programs to cgroups: create cgroup program: load program: invalid argument: not supported” subsys=daemon
level=error msg=“Start hook failed” error=“daemon creation failed: unable to initialize kube-proxy replacement options: socketlb enabled, but kernel does not support attaching bpf programs to cgroups: create cgroup program: load program: invalid argument: not supported” function=cell.augmentedHook.Start subsys=hive
level=info msg=Stopping subsys=hive
level=info msg=“Stop hook executed” duration=“1.6µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“22.593µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“1.472µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“76.579µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“1.088µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“38.082µs” function=cell.augmentedHook.Stop subsys=hive
level=error msg=“Observer job stopped with an error” error=“context canceled” func=“auth.(*AuthManager).handleAuthRequest” name=“auth request-authentication” subsys=auth
level=error msg=“Observer job stopped with an error” error=“context canceled” func=“auth.(*authMapGarbageCollector).handleIdentityChange” name=“auth gc-identity-events” subsys=auth
level=info msg=“Stop hook executed” duration=“374.482µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“9.44µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=736ns function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Datapath signal listener exiting” subsys=signal
level=info msg=“Datapath signal listener done” subsys=signal
level=info msg=“Stop hook executed” duration=“530.584µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“49.027µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“326.031µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=800ns function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“20.961µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“16.193µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“40.865µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“15.585µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“4.128µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“3.904µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“325.295µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“38.337µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“134.502µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“15.297µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“23.809µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“22.561µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“72.516µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“21.601µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“19.425µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“65.027µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“127.973µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=7.6289ms function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“72.899µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=11.894826ms function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“147.271µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“1.28µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“20.449µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“14.625µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“11.425µs” function=cell.augmentedHook.Stop subsys=hive
level=error msg=“Close() called without calling InitIdentityAllocator() first” subsys=identity-cache
level=error msg=“Stop hook failed” error=“unable to find controller ipcache-inject-labels” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“40.418µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“47.074µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“2.144µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“1.504µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“2.4µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“213.482µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“26.177µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“33.41µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“6.016µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“41.826µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“66.179µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“240.619µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“102.085µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“3.232µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“91.909µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“29.409µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“31.809µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“26.817µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“97.509µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stop hook executed” duration=“239.883µs” function=cell.augmentedHook.Stop subsys=hive
level=info msg=“Stopped gops server” address=“127.0.0.1:9890” subsys=gops
level=info msg=“Stop hook executed” duration=“808.518µs” function=cell.augmentedHook.Stop subsys=hive
level=fatal msg=“failed to start: daemon creation failed: unable to initialize kube-proxy replacement options: socketlb enabled, but kernel does not support attaching bpf programs to cgroups: create cgroup program: load program: invalid argument: not supported\nfailed to stop: unable to find controller ipcache-inject-labels” subsys=daemon