I am currently working on building a completely customized rootfs following your documentation. I have managed to do that successfully using the QEMU emulator, creating the entire setup on an x86 host machine for my board.
I am sure you know how slow the QEMU emulator is compared with a native ARM64 machine. I have tried building the same setup, with minor adjustments of course, on my MacBook M2. Unfortunately, there are scripts that require QEMU and in case it does not exist, they fail, e.g.:
cat nv-apply-debs.sh
...
if [ -z "${QEMU_BIN}" ]; then
echo "ERROR qemu not found! To install - please run: " \
"\"sudo apt-get install qemu-user-static\""
exit 1
fi
...
cat l4t_create_default_user.sh
...
if [ ! -f "/usr/bin/qemu-aarch64-static" ]; then
echo "ERROR: please install qemu-user-static package" > /dev/stderr
usage
fi
...
(I assume that there are more).
Is there a way to overcome this issue? I suspect that building on a native ARM64 machine would generate faster results which would be worth a lot of $ in the end.
I can install it, but not sure if I need it. I’m compiling everything on a ARM64 machine (Macbook M2) already, why would I need QEMU in that context?
the point is that even though you have a native ARM machine, and do not have to use QEMU to emulate an AArch64 architecture, binary formats on Mac and Linux are still not the same (Mach-O/ELF), so you cannot run executables (like dkpg for extracting packages) in the BSP’s rootfs.
In short, I think it’s possible to do on an ARM Linux, but not on an ARM Mac.
I don’t know much about docker, but in theory, you wouldn’t need QEMU if your docker is correct. However, the l4t_create_default_user.sh uses QEMU and expects to be run from an amd64/x86_64 Linux host PC in order to execute ARMv8-a code. Going from Mac to Linux on Mac might differ versus Linux on an amd64/x86_64 regular PC. I can see how Mac and regular PC might be no different if the Mac has the amd64/x86_64 CPU, but if the Linux itself is any different for installing to a Mac, then you would need to emulate a PC in order to run that script from a PC. Does docker look like a Mac variant amd64/x86_64, or is it an exact match? Even if it does, then you still need the static because a PC needs the static QEMU to pretend to be ARMv8-a.
I think you may do it using Docker (though you’d still need to modify the script to bypass QEMU stuff), but Docker on Mac are done via virtual machines, not like the container mechanism on Linux, so there will be degradation in performance, which is also seen on x86 with QEMU.
Given that, it may not bring that much benefits even if you manage to run it on your ARM Mac, and another thing to note is that you will eventually need a x86 Ubuntu PC to flash your devices, then why not just do it all on your PC?
[quote=“DaveYYY, post:8, topic:253883, full:true”]
I think you may do it using Docker (though you’d still need to modify the script to bypass QEMU stuff), [/quote]
So currently there’s no way to do that without QEMU, right? What if I wanted to build it directly on a Jetson?
Probably, but not as bad as QEMU.
Let’s differentiate between building the image and flashing it. My end goal is to build the image using a CI on ARM64 agents (probably a Macbook Github Action runners, but it can also be AWS ARM64 servers and then download it on a host machine and flash it.
It should match. Using M1/M2 runners/agents in a CI is becoming trendy (and very fast)
So currently there’s no way to do that without QEMU, right? What if I wanted to build it directly on a Jetson?
I think it’s feasible on ARM Linux/Jetson, but we don’t have the experience doing so. You may try if removing copying the QEMU executables from the scripts works.