Docker image for cross compilation?

Hi,

I have no experience in cross compiling (well, other then android but thats a bit different).
Is there a docker image that allows us to easily cross compile for the nano? The whole setup is a bit scary and I do not want to “contaminate” my production server.

I don’t know of a Docker image that will do it, but these instructions worked for me to build the MMAPI samples. I modified them slightly, however, using the sshfs fuse (user space) driver to mount instead of samba (and I didn’t have to set up samba or modify the nano at all other than to enable ssh temporarily). Archivemount of a rootfs tarball works as well, but in testing it was way too slow to be useful.

Basically the idea is to have a mountpoint where a rootfs can be found. That’s defined in the TARGET_ROOTFS environment variable. Then you set CROSS_COMPILE to define the toolchain prefix. Provided you installed JetPack with the SDK manager on your Ubuntu host, you should have the following files:

[user@hostname] -- [~/Dev/nano] 
 $ ls /usr/bin/aarch64-linux-gnu-*
/usr/bin/aarch64-linux-gnu-addr2line     /usr/bin/aarch64-linux-gnu-gcov
/usr/bin/aarch64-linux-gnu-ar            /usr/bin/aarch64-linux-gnu-gcov-7
/usr/bin/aarch64-linux-gnu-as            /usr/bin/aarch64-linux-gnu-gcov-dump
/usr/bin/aarch64-linux-gnu-c++filt       /usr/bin/aarch64-linux-gnu-gcov-dump-7
/usr/bin/aarch64-linux-gnu-cpp           /usr/bin/aarch64-linux-gnu-gcov-tool
/usr/bin/aarch64-linux-gnu-cpp-7         /usr/bin/aarch64-linux-gnu-gcov-tool-7
/usr/bin/aarch64-linux-gnu-dwp           /usr/bin/aarch64-linux-gnu-gprof
/usr/bin/aarch64-linux-gnu-elfedit       /usr/bin/aarch64-linux-gnu-ld
/usr/bin/aarch64-linux-gnu-g++           /usr/bin/aarch64-linux-gnu-ld.bfd
/usr/bin/aarch64-linux-gnu-g++-7         /usr/bin/aarch64-linux-gnu-ld.gold
/usr/bin/aarch64-linux-gnu-gcc           /usr/bin/aarch64-linux-gnu-nm
/usr/bin/aarch64-linux-gnu-gcc-7         /usr/bin/aarch64-linux-gnu-objcopy
/usr/bin/aarch64-linux-gnu-gcc-ar        /usr/bin/aarch64-linux-gnu-objdump
/usr/bin/aarch64-linux-gnu-gcc-ar-7      /usr/bin/aarch64-linux-gnu-ranlib
/usr/bin/aarch64-linux-gnu-gcc-nm        /usr/bin/aarch64-linux-gnu-readelf
/usr/bin/aarch64-linux-gnu-gcc-nm-7      /usr/bin/aarch64-linux-gnu-size
/usr/bin/aarch64-linux-gnu-gcc-ranlib    /usr/bin/aarch64-linux-gnu-strings
/usr/bin/aarch64-linux-gnu-gcc-ranlib-7  /usr/bin/aarch64-linux-gnu-strip

so CROSS_COMPILE should be “/usr/bin/aarch64-linux-gnu-”

To automate things you can have a script like this to activate your environment:

export TARGET_ROOTFS=$HOME/Dev/nano/rootfs_mountpoint
export CROSS_COMPILE=/usr/bin/aarch64-linux-gnu-

And you just do <b>source</b> scriptnamehere at a prompt to set the environment variables in that shell. Then, in my (admittedly limited) experience, it just builds:

From start to finish:

[username@hostname] -- [~/Dev/nano] 
 $ source activate_cross_compile 
/usr/bin/aarch64-linux-gnu-
[username@hostname] -- [~/Dev/nano] 
 $ cd src/tegra_multimedia_api/
[username@hostname] -- [~/Dev/nano/src/tegra_multimedia_api] 
 $ make clean && make -j8
Clean in samples/00_video_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/00_video_decode'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/00_video_decode'
Clean in samples/01_video_encode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/01_video_encode'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/01_video_encode'
Clean in samples/02_video_dec_cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/02_video_dec_cuda'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/02_video_dec_cuda'
Clean in samples/03_video_cuda_enc
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/03_video_cuda_enc'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/03_video_cuda_enc'
Clean in samples/04_video_dec_trt
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/04_video_dec_trt'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/04_video_dec_trt'
Clean in samples/05_jpeg_encode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/05_jpeg_encode'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/05_jpeg_encode'
Clean in samples/06_jpeg_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/06_jpeg_decode'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/06_jpeg_decode'
Clean in samples/07_video_convert
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/07_video_convert'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/07_video_convert'
Clean in samples/08_video_dec_drm
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/08_video_dec_drm'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/08_video_dec_drm'
Clean in samples/09_camera_jpeg_capture
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/09_camera_jpeg_capture'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/09_camera_jpeg_capture'
Clean in samples/10_camera_recording
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/10_camera_recording'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/10_camera_recording'
Clean in samples/12_camera_v4l2_cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/12_camera_v4l2_cuda'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/12_camera_v4l2_cuda'
Clean in samples/13_multi_camera
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/13_multi_camera'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/13_multi_camera'
Clean in samples/14_multivideo_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/14_multivideo_decode'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/14_multivideo_decode'
Clean in samples/backend
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/backend'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/backend'
Clean in samples/frontend
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/frontend'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/frontend'
Clean in samples/v4l2cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/v4l2cuda'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/v4l2cuda'
Clean in tools/ConvertCaffeToTrtModel
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/tools/ConvertCaffeToTrtModel'
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/tools/ConvertCaffeToTrtModel'
Make in samples/00_video_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/00_video_decode'
Compiling: video_decode_csvparser.cpp
Compiling: video_decode_main.cpp
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
Compiling: NvElementProfiler.cpp
Compiling: NvElementProfiler.cpp
Compiling: NvElementProfiler.cpp
Compiling: NvElementProfiler.cpp
Compiling: NvElementProfiler.cpp
Compiling: NvElementProfiler.cpp
Compiling: NvElement.cpp
Compiling: NvElement.cpp
Compiling: NvElement.cpp
Compiling: NvElement.cpp
Compiling: NvElement.cpp
Compiling: NvElement.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvBuffer.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvBuffer.cpp
Compiling: NvLogging.cpp
Compiling: NvBuffer.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvLogging.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvLogging.cpp
Compiling: NvUtils.cpp
Compiling: NvBuffer.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2Element.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2Element.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2Element.cpp
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
Compiling: NvV4l2Element.cpp
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
Compiling: NvV4l2Element.cpp
Compiling: NvV4l2Element.cpp
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
Compiling: NvV4l2Element.cpp
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/classes'
Linking: video_decode
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/00_video_decode'
Make in samples/01_video_encode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/01_video_encode'
Compiling: video_encode_csvparser.cpp
Compiling: video_encode_main.cpp
Linking: video_encode
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/01_video_encode'
Make in samples/02_video_dec_cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/02_video_dec_cuda'
Compiling: videodec_csvparser.cpp
Compiling: videodec_main.cpp
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/cuda'
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/cuda'
Compiling: NvAnalysis.cu
Compiling: NvAnalysis.cu
Compiling: NvCudaProc.cpp
Compiling: NvCudaProc.cpp
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/cuda'
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/cuda'
Linking: video_dec_cuda
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/02_video_dec_cuda'
Make in samples/03_video_cuda_enc
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/03_video_cuda_enc'
Compiling: video_cuda_enc_csvparser.cpp
Compiling: video_cuda_enc_main.cpp
Linking: video_cuda_enc
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/03_video_cuda_enc'
Make in samples/04_video_dec_trt
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/04_video_dec_trt'
Compiling: video_dec_trt_csvparser.cpp
Compiling: video_dec_trt_main.cpp
make[2]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/trt'
Compiling: trt_inference.cpp
make[2]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/common/algorithm/trt'
Linking: video_dec_trt
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/04_video_dec_trt'
Make in samples/05_jpeg_encode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/05_jpeg_encode'
Compiling: jpeg_encode_csvparser.cpp
Compiling: jpeg_encode_main.cpp
Linking: jpeg_encode
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/05_jpeg_encode'
Make in samples/06_jpeg_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/06_jpeg_decode'
Compiling: jpeg_decode_csvparser.cpp
Compiling: jpeg_decode_main.cpp
Linking: jpeg_decode
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/06_jpeg_decode'
Make in samples/07_video_convert
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/07_video_convert'
Compiling: video_convert_csvparser.cpp
Compiling: video_convert_main.cpp
Linking: video_convert
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/07_video_convert'
Make in samples/08_video_dec_drm
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/08_video_dec_drm'
/usr/bin/python /home/username/Dev/nano/src/tegra_multimedia_api/tools/GetPixel.py -i /home/username/Dev/nano/src/tegra_multimedia_api/data/Picture/nvidia-logo.jpg -o image_rgba.cpp
Compiling: video_dec_drm_csvparser.cpp
Compiling: video_dec_drm.cpp
WARN: don't error out if PIL is not installed, instead to create a color pattern.
Alternatively, you can install it by <sudo apt-get install python-imaging>
Compiling: image_rgba.cpp
Linking: video_dec_drm
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/08_video_dec_drm'
Make in samples/09_camera_jpeg_capture
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/09_camera_jpeg_capture'
Compiling: main.cpp
Compiling: /home/username/Dev/nano/src/tegra_multimedia_api/argus/samples/utils/Thread.cpp
Linking: camera_jpeg_capture
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/09_camera_jpeg_capture'
Make in samples/10_camera_recording
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/10_camera_recording'
Compiling: main.cpp
Compiling: /home/username/Dev/nano/src/tegra_multimedia_api/argus/samples/utils/NativeBuffer.cpp
Compiling: /home/username/Dev/nano/src/tegra_multimedia_api/argus/samples/utils/nvmmapi/NvNativeBuffer.cpp
Linking: camera_recording
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/10_camera_recording'
Make in samples/12_camera_v4l2_cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/12_camera_v4l2_cuda'
Compiling: camera_v4l2_cuda.cpp
Linking: camera_v4l2_cuda
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/12_camera_v4l2_cuda'
Make in samples/13_multi_camera
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/13_multi_camera'
Compiling: main.cpp
Linking: multi_camera
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/13_multi_camera'
Make in samples/14_multivideo_decode
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/14_multivideo_decode'
Compiling: multivideo_decode_csvparser.cpp
Compiling: multivideo_decode_main.cpp
Linking: multivideo_decode
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/14_multivideo_decode'
Make in samples/backend
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/backend'
Compiling: v4l2_backend_csvparser.cpp
Compiling: v4l2_backend_main.cpp
Linking: backend
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/backend'
Make in samples/frontend
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/frontend'
Compiling: main.cpp
Compiling: StreamConsumer.cpp
Compiling: VideoEncodeStreamConsumer.cpp
Compiling: VideoEncoder.cpp
Compiling: TRTStreamConsumer.cpp
Linking: frontend
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/frontend'
Make in samples/v4l2cuda
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/v4l2cuda'
Compiling: capture.cpp
Compiling: yuv2rgb.cu
Linking: capture-cuda
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/samples/v4l2cuda'
Make in tools/ConvertCaffeToTrtModel
make[1]: Entering directory '/home/username/Dev/nano/src/tegra_multimedia_api/tools/ConvertCaffeToTrtModel'
Compiling: ConvertCaffeToTrtModel_main.cpp
Linking: ConvertCaffeToTrtModel
make[1]: Leaving directory '/home/username/Dev/nano/src/tegra_multimedia_api/tools/ConvertCaffeToTrtModel'

Frighteningly fast compared to the Nano.

I think each build system will handle things a little differently. I am new to cross-compilation as well and have never tested these (soon), but here are some instructions for cmake you might be able to adapt with the above idea.

Thank you, spending 24h compiling pytorch, I think probably anything would be frighteningly faster :)
however, my pristine server is not to be messed with, so I am still looking for docker install. Perhaps NVIDIA have a dockerfile… who knows…

No kidding :) If it takes a day to compile, I’d rather spend spend the day learning how to cross compile and finish the actual build in minutes. The OpenCV builds were absurd.

You could install Ubuntu on a virtual machine and build within that. Do you have a desktop with virtualisation extensions and Windows Pro? If you use Docker Desktop the answer is probably yes. HyperV manager is great.

TBH i had to restart the compilation twice because of swap/disk space problems
would still take over 12h
mmmm… I could probably crank up ubuntu docker image and use commit after installing everything. Would still be 10000 times nicer to just have a preset docker

my server is ubuntu. headless… I am reluctant to install anything on it as it is used for trainning models.

I guess I’m missing something, but is your server your only box apart from the Nano? If you have a Windows Pro desktop you can install HyperV on (part of windows) and build from within a VM (or Docker)? If you have Linux running on the desktop, that’s even easier.

no, i have many laptops but they are not remotely as powerful as this one or dedicated to a specfic task as well.

It’s as you prefer. I use a NUC running Linux to build (with some nice specs, particularly ram). You may find it worth it except maybe on a netbook with very limited specs.

Cross compile is faster, but if you strictly wanted to use a docker image, here’s an example:

Dockerfile

FROM balenalib/aarch64-ubuntu:latest

RUN [ "cross-build-start" ]

# ADD ALL YOUR STEPS HERE 
# AS IF YOU WERE MAKING A 
# TRADITIONAL DOCKER IMAGE

RUN [ "cross-build-end" ]

This will produce a docker image that can be built on x86 and will run on the nano. If you want to use this same technique as a “cross compiler”, you can elect to not call the cross-build-end script, and the image will be runnable on x86, but will execute everything under a qemu-arm process, effectively running an “emulated” aarch64 container. Then you could do something like:

Dockerfile

FROM balenalib/aarch64-ubuntu:latest

RUN [ "cross-build-start" ]

# INSTALL ANY TOOLCHAINS YOU NEED

ENTRYPOINT [ "/bin/bash" ]

Build via

docker build -t MyImage .

call it with

docker run --rm -it -v my_folder_that_has_code:/src MyImage

Which will open up a bash prompt within the emulated aarch64 container. The -v flag mounted your folder which you wanted to compile, so follow your standard compile instructions from within the container’s /src directory and it will produce aarch64 versions of your software. Note that this assumes you’ve installed all the necessary toolchains needed to compile your target within the container (where I marked in the dockerfile).

For context, see the balena blog for more info on what they are doing behind the scenes to make this work (it’s pretty slick): Building ARM containers on any x86 machine, even DockerHub

no no no

I want a docker image that runs on the x86 and has everything you need to cross compile FOR the nano.

see examples GitHub - dockcross/dockcross: Cross compiling toolchains in Docker images

That’s what the image does, Moshe. The only problem with it I can see is that it lacks Nvidia libraries and headers (but those could be added). If you’re building a package or Docker arm64 image that doesn’t depend on those, it a really good solution, however.

Edit: I found Jetson images from balena for tx1 and tx2. Those should work out of the box, I would think.

Edit: hit wrong button on mobile.

this has confused me “This will produce a docker image that can be built on x86 and will run on the nano”

I see… this might just work. I am not using currently any special libs but if I’ll go the tensorrt way, I guess I’ll need to install something for it.

Thank you!

BTW, this is still emulation. Would be far far better if I could have an image like the ones in the docker repository for cross compiling with all libraries installed etc.

Yes, it’s emulation, so it’ll be slower, but it’s a really elegant solution, and probably still a lot faster than the Nano. If you want to do a containerized cross compile, You might be better off making your own Dockerfiles per project (or a set of templates) to accomplish that. Each project probably has different build and runtime dependencies after all. I am going to experiment myself and if I make any interesting Dockerfiles, I will post them here.

You are right about libraries etc, but a working template to start from would be awesome. Tried doing it myself but was overwhelmed by the documentation and my lack of understanding of the process.

Moshe,

I found some really great instructions:

If anything, this is slightly discouraging… the amount of problems he encountered is scary.
Anyways, I would assume this was actually used to created the github images (as this is from 2015). I am waiting for the server to finish something and then i’ll using the github image. I am slightly skeptical if this can actually compile something like pytorch… I would say only proper cross compilation would work properly, with the target file system mounted, but again, this is all witchcraft for me.