nvcr.io/nvidia/tensorflow:X containers have too many layers

Is it possible to reduce the number of layers of nvcr.io/nvidia/tensorflow container images ? The 19.09-py2 has 115 layers. That limits the number of additional layers that can be added.

The tensorflow/tensorflow:latest-gpu images only have 30 odd layers.

Here are layers of the 19.09-py2 image. Are all these layers necessary ?

IMAGE CREATED CREATED BY SIZE COMMENT
b82bcb185286 13 days ago /bin/sh -c #(nop) LABEL com.nvidia.build.re… 0B
13 days ago /bin/sh -c #(nop) ARG NVIDIA_BUILD_REF 0B
13 days ago /bin/sh -c #(nop) LABEL com.nvidia.build.id… 0B
13 days ago /bin/sh -c #(nop) ENV NVIDIA_BUILD_ID=80447… 0B
13 days ago /bin/sh -c #(nop) ARG NVIDIA_BUILD_ID 0B
13 days ago |1 TFAPI=1 /bin/sh -c ln -sf ${CUDA_COMPAT… 46.4kB
13 days ago /bin/sh -c #(nop) ENV PATH=/usr/local/mpi/b… 0B
13 days ago |1 TFAPI=1 /bin/sh -c cd /opt/tensorrt/pytho… 1.13MB
13 days ago /bin/sh -c #(nop) COPY file:73c40c29e38bdb61… 490B
13 days ago |1 TFAPI=1 /bin/sh -c BASE=0 DEVEL=0 SAMPLES… 31.9MB
13 days ago /bin/sh -c #(nop) COPY dir:bb39c1bc17bc0c2d0… 38.8kB
13 days ago |1 TFAPI=1 /bin/sh -c cd /workspace/nvidia-e… 36.4MB
13 days ago /bin/sh -c #(nop) COPY file:63121bf68711a432… 269B
13 days ago |1 TFAPI=1 /bin/sh -c pip install --no-cache… 299MB
13 days ago |1 TFAPI=1 /bin/sh -c cd /workspace/nvidia-e… 415kB
13 days ago |1 TFAPI=1 /bin/sh -c PROTOC_VERSION=3.0.0 &… 4.1MB
13 days ago |1 TFAPI=1 /bin/sh -c pip install --no-cache… 7.6MB
13 days ago |1 TFAPI=1 /bin/sh -c pip install --no-cache… 9.54MB
13 days ago |1 TFAPI=1 /bin/sh -c cd /workspace/nvidia-e… 123MB
13 days ago /bin/sh -c #(nop) COPY dir:1bfd7891d5a65bf0d… 701MB
13 days ago /bin/sh -c #(nop) COPY dir:a9d1446c5e6418c10… 1.63kB
13 days ago /bin/sh -c #(nop) COPY file:75def6eead007f70… 2.98kB
13 days ago |1 TFAPI=1 /bin/sh -c chmod -R a+w /workspace 0B
13 days ago /bin/sh -c #(nop) WORKDIR /workspace 0B
13 days ago /bin/sh -c #(nop) ENTRYPOINT ["/usr/local/b… 0B
13 days ago /bin/sh -c #(nop) COPY file:8601f05150a54f40… 3.98kB
13 days ago |1 TFAPI=1 /bin/sh -c pip install --no-cache… 132MB
13 days ago /bin/sh -c #(nop) ENV DALI_VERSION=0.13.0 D… 0B
13 days ago |1 TFAPI=1 /bin/sh -c export HOROVOD_GPU_ALL… 61.9MB
13 days ago /bin/sh -c #(nop) COPY dir:726130932e2cae4c8… 46.7MB
13 days ago /bin/sh -c #(nop) ENV TF_ADJUST_HUE_FUSED=1… 0B
13 days ago |1 TFAPI=1 /bin/sh -c /opt/tensorflow/nvbuil… 2.09GB
13 days ago /bin/sh -c #(nop) ARG TFAPI=1 0B
13 days ago /bin/sh -c #(nop) COPY file:bb5c2065dfff2880… 38.7kB
13 days ago /bin/sh -c #(nop) COPY multi:03065681f6ea6a3… 7.63kB
13 days ago /bin/sh -c #(nop) COPY dir:95d2d3cf607a4b216… 162MB
2 weeks ago /bin/sh -c #(nop) EXPOSE 6064 0B
2 weeks ago /bin/sh -c #(nop) EXPOSE 6006 0B
2 weeks ago /bin/sh -c #(nop) EXPOSE 8888 0B
2 weeks ago /bin/sh -c #(nop) ENV TENSORBOARD_DEBUGGER_… 0B
2 weeks ago /bin/sh -c #(nop) ENV TENSORBOARD_PORT=6006 0B
2 weeks ago /bin/sh -c #(nop) ENV JUPYTER_PORT=8888 0B
2 weeks ago /bin/sh -c mv /root/.jupyter/jupyter_noteboo… 122B
2 weeks ago /bin/sh -c #(nop) COPY file:afbf9a402a0f9d49… 427B
2 weeks ago /bin/sh -c source “$NVM_DIR/nvm.sh” && pip … 6.26MB
2 weeks ago /bin/sh -c pip install --no-cache-dir jupyte… 85.7MB
2 weeks ago /bin/sh -c #(nop) ENV NVM_DIR=/usr/local/nvm 0B
2 weeks ago /bin/sh -c TENSORBOARD_VERSION=1.14.0+nv … 28.8MB
2 weeks ago /bin/sh -c DLPROF_VERSION=0.4.0_19.09 && mk… 47.6MB
2 weeks ago /bin/sh -c MKL_DNN_COMMIT=733fc908874c71a528… 1.43MB
2 weeks ago /bin/sh -c #(nop) COPY file:21ab1ea8047db2f9… 511B
2 weeks ago /bin/sh -c #(nop) WORKDIR /opt/tensorflow 0B
2 weeks ago /bin/sh -c BAZEL_VERSION=0.24.1 && mkdir… 98.2MB
2 weeks ago /bin/sh -c echo “startup --batch” >> $BAZELR… 106B
2 weeks ago /bin/sh -c #(nop) ENV BAZELRC=/root/.bazelrc 0B
2 weeks ago /bin/sh -c pip install --no-cache-dir --upgr… 195MB
2 weeks ago /bin/sh -c curl -O https://bootstrap.pypa.io… 13.6MB
2 weeks ago /bin/sh -c mkdir -p /usr/lib/x86_64-linux-gn… 20B
2 weeks ago /bin/sh -c rm -f /usr/bin/python && rm -… 36B
2 weeks ago /bin/sh -c #(nop) ENV PYTHONIOENCODING=utf-… 0B
2 weeks ago /bin/sh -c PYSFX=[ "$PYVER" != "2.7" ] && e… 267MB <missing> 2 weeks ago /bin/sh -c #(nop) ENV PYVER=2.7 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG PYVER=3.6 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV NVIDIA_TENSORFLOW_VER… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG NVIDIA_TENSORFLOW_VER… 0B <missing> 2 weeks ago /bin/sh -c #(nop) LABEL com.nvidia.tensorfl… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV TENSORFLOW_VERSION=1.… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV NVM_DIR=/usr/local/nvm 0B <missing> 2 weeks ago /bin/sh -c TENSORBOARD_VERSION=1.14.0+nv … 28.8MB <missing> 2 weeks ago /bin/sh -c DLPROF_VERSION=0.4.0_19.09 && mk… 47.6MB <missing> 2 weeks ago /bin/sh -c MKL_DNN_COMMIT=733fc908874c71a528… 1.43MB <missing> 2 weeks ago /bin/sh -c #(nop) COPY file:21ab1ea8047db2f9… 511B <missing> 2 weeks ago /bin/sh -c #(nop) WORKDIR /opt/tensorflow 0B <missing> 2 weeks ago /bin/sh -c BAZEL_VERSION=0.24.1 && mkdir… 98.2MB <missing> 2 weeks ago /bin/sh -c echo "startup --batch" >> $BAZELR… 106B <missing> 2 weeks ago /bin/sh -c #(nop) ENV BAZELRC=/root/.bazelrc 0B <missing> 2 weeks ago /bin/sh -c pip install --no-cache-dir --upgr… 195MB <missing> 2 weeks ago /bin/sh -c curl -O https://bootstrap.pypa.io… 13.6MB <missing> 2 weeks ago /bin/sh -c mkdir -p /usr/lib/x86_64-linux-gn… 20B <missing> 2 weeks ago /bin/sh -c rm -f /usr/bin/python && rm -… 36B <missing> 2 weeks ago /bin/sh -c #(nop) ENV PYTHONIOENCODING=utf-… 0B <missing> 2 weeks ago /bin/sh -c PYSFX=[ “PYVER" != "2.7" ] && e… 267MB <missing> 2 weeks ago /bin/sh -c #(nop) ENV PYVER=2.7 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG PYVER=3.6 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV NVIDIA_TENSORFLOW_VER… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG NVIDIA_TENSORFLOW_VER… 0B <missing> 2 weeks ago /bin/sh -c #(nop) LABEL com.nvidia.tensorfl… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV TENSORFLOW_VERSION=1.… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG TENSORFLOW_VERSION 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV LD_LIBRARY_PATH=/usr/… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV LIBRARY_PATH=/usr/loc… 0B <missing> 2 weeks ago /bin/sh -c export DEVEL=1 BASE=0 && /nvidia… 1.08GB <missing> 2 weeks ago /bin/sh -c #(nop) COPY file:fcaa53ecffdb0085… 1.49kB <missing> 2 weeks ago /bin/sh -c #(nop) COPY dir:d17c6a0bb7977f553… 38.8kB <missing> 2 weeks ago /bin/sh -c #(nop) ENV OMPI_MCA_btl_vader_si… 0B <missing> 2 weeks ago /bin/sh -c #(nop) ENV PATH=/usr/local/mpi/b… 0B <missing> 2 weeks ago /bin/sh -c wget -q -O - https://www.open-mpi… 13.2MB <missing> 2 weeks ago /bin/sh -c #(nop) ENV OPENMPI_VERSION=3.1.4 0B <missing> 2 weeks ago /bin/sh -c #(nop) ARG OPENMPI_VERSION=3.1.4 0B <missing> 2 weeks ago /bin/sh -c dpkg -i /opt/mellanox/DEBS/{MOFE… 2.02MB
2 weeks ago /bin/sh -c #(nop) ENV MOFED_VERSION=4.4-1.0… 0B
2 weeks ago /bin/sh -c #(nop) ARG MOFED_VERSION=4.4-1.0… 0B
2 weeks ago /bin/sh -c #(nop) COPY dir:e22206b8fba8289c2… 5.39MB
2 weeks ago /bin/sh -c export DEBIAN_FRONTEND=noninterac… 93MB
2 weeks ago /bin/sh -c #(nop) COPY file:98526337940e33de… 1.19kB
2 weeks ago /bin/sh -c #(nop) COPY file:e475a93e37fc1b0b… 624kB
2 weeks ago /bin/sh -c #(nop) ENV PATH=/usr/local/nvidi… 0B
2 weeks ago /bin/sh -c echo “/usr/local/nvidia/lib” >> /… 46B
2 weeks ago /bin/sh -c #(nop) ADD file:6e2f1800f7a39ebee… 13.4kB
2 weeks ago /bin/sh -c rm -rf /nvidia 0B
2 weeks ago /bin/sh -c #(nop) LABEL com.nvidia.nccl.ver… 0B
2 weeks ago /bin/sh -c /nvidia/build-scripts/installNCCL… 896MB
2 weeks ago /bin/sh -c #(nop) ENV NCCL_VERSION=2.4.8 CU… 0B
2 weeks ago /bin/sh -c #(nop) ARG NSIGHT_SYSTEMS_VERSION 0B
2 weeks ago /bin/sh -c #(nop) ARG TRT_VERSION 0B
2 weeks ago /bin/sh -c #(nop) ARG CUDNN_VERSION 0B
2 weeks ago /bin/sh -c #(nop) ARG CUBLAS_VERSION 0B
2 weeks ago /bin/sh -c #(nop) ARG NCCL_VERSION 0B
2 weeks ago /bin/sh -c #(nop) LABEL com.nvidia.volumes.… 0B
2 weeks ago /bin/sh -c #(nop) ENV _CUDA_COMPAT_PATH=/us… 0B
2 weeks ago /bin/sh -c patch -p0 < /tmp/startup_scripts.… 3.19kB
2 weeks ago /bin/sh -c #(nop) COPY file:f6bcc0ba042461fc… 1.33kB
2 weeks ago /bin/sh -c #(nop) COPY file:89b70e66afee46dc… 13.4kB
2 weeks ago /bin/sh -c #(nop) COPY file:2008ae75c542d782… 8.77kB
2 weeks ago /bin/sh -c /nvidia/build-scripts/installCUDA… 1.25GB
2 weeks ago /bin/sh -c #(nop) ENV CUDA_VERSION=10.1.243… 0B
2 weeks ago /bin/sh -c #(nop) ARG CUDA_DRIVER_VERSION 0B
2 weeks ago /bin/sh -c #(nop) ARG CUDA_VERSION 0B
2 weeks ago /bin/sh -c #(nop) COPY dir:d17c6a0bb7977f553… 38.8kB
5 weeks ago /bin/sh -c export DEBIAN_FRONTEND=noninterac… 20.6MB
6 weeks ago /bin/sh -c #(nop) CMD [”/bin/bash"] 0B
6 weeks ago /bin/sh -c mkdir -p /run/systemd && echo 'do… 7B
6 weeks ago /bin/sh -c set -xe && echo ‘#!/bin/sh’ > /… 745B
6 weeks ago /bin/sh -c [ -z “$(apt-get indextargets)” ] 987kB
6 weeks ago /bin/sh -c #(nop) ADD file:c477cb0e95c56b51e… 63.2MB

Yes - the feedback is helpful. How many layers are you looking to add and what is the number of layer limit that you’re encountering?

Thanks

Joey

I think the max on Docker is 190 layers. Most derived containers for apps/dev/research adds layers - 100+ .