Run a 32bit Application on the Jetson Nano

Update:
Seems like the process cannot be applied to all libraries. Installing some libraries (32 bit) will result in removal of 64 bit counterparts including dependencies. So there is no real 32 bit subsystem possible for Linux ARM.

If you specifically marked armhf as a foreign architecture, and specifically go to install the 32-bit version, then it might let you install without removing the other packages. Are you trying to install while naming the actual package architecture, and not just the generic package name? For example, naming architecture:
sudo apt-get install libappindicator3-1:armhf

I am guessing you did, but if it is a simple case of just naming the architecture, then it is worth mentioning.

Yes. I did select the armhf version of the library in Synaptic. On the command I get a different message

rreddy78@jetson:~/$ sudo apt-get install libappindicator3-1:armhf

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libappindicator3-1:armhf : Depends: libindicator3-7:armhf (>= 0.4.90) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
rreddy78@jetson:~/$ sudo apt-get install libappindicator3-7:armhf
Reading package lists… Done
Building dependency tree
Reading state information… Done
E: Unable to locate package libappindicator3-7:armhf
rreddy78@jetson:~/$

I am thinking this might not be an issue of armhf foreign versus arm64 native. What do you see from:
dpkg -l | grep -i 'libindicator'

If you happen to have a version of libindicator already installed, but it is too new, then you might be able to remove that, and search for a “compat” version (not sure if I can search specifically for armhf, so listing two searches…one might not be a real command…my Nano is not set up with armhf so I can’t really test):

apt search libindicator
apt search libindicator3
apt search libindicator3-7
apt search libindicator3-7:armhf

Basically I am thinking that if there is an out of date release version, and if there is a newer or compat version of libindicator which is version 0.4.90 or greater, then you could get rid of whatever the current version is and then you could fulfill “Depends: libindicator3-7:armhf (>= 0.4.90) but it is not going to be installed”. This might be completely unrelated to the armhf being foreign. Just make sure you only remove an out of date armhf version, and not the arm64 version.

rreddy78@jetson:~$ dpkg -l | grep -i 'libindicator'
ii  libindicator3-7                                 16.10.0+18.04.20180321.1-0ubuntu1                   arm64        panel indicator applet - shared library
libindicator-dev/bionic 16.10.0+18.04.20180321.1-0ubuntu1 arm64
  panel indicator applet - library development files

libindicator3-7/bionic,now 16.10.0+18.04.20180321.1-0ubuntu1 arm64 [installed,automatic]
  panel indicator applet - shared library

libindicator3-dev/bionic 16.10.0+18.04.20180321.1-0ubuntu1 arm64
  panel indicator applet - library development files

libindicator3-tools/bionic 16.10.0+18.04.20180321.1-0ubuntu1 arm64
  Tools for libindicator

libindicator7/bionic 16.10.0+18.04.20180321.1-0ubuntu1 arm64
  panel indicator applet - shared library

rreddy78@jetson:~$ apt search libindicator3-7:armhf
Sorting... Done
Full Text Search... Done
rreddy78@jetson:~$ 

But there is some good news: the Docker approach works better!

I was trying to get the Chromium 32bit version with Widevine CDM working on my system with the method given here:

Gist:
https://gist.github.com/teacupx

I used this :

I can send my modified dockerfile if anyone needs…

And i am able to run Chromium 32bit
However:

  1. WideCDM still not working. Still investigating…
  2. Cannot use hw acceleration. Fails at glInitialize (looks like there can be only one GL library running at a time…). Still investigating…

If you want to use GPU HW acceleration, try using virglrenderer with Xephyr as display, I’m combining it with armhf chroot instead of docker. Only issue is that GPU is not going turbo this way (40% usage @ 230 MHz running glmark2, even lower in Gothic using box86). I don’t know why…

Thanks

How is this done ?

I failed to get Chromium working with widevine. Apparently some additional patching required. However I could enable it for the vivaldi browser with this script

A script that fetches a ChromeOS image for ARM32 and extracts the Widevine binary, saving it in a compressed archive for use with Vivaldi · GitHub

I think the best way would be to have a armhf QEMU VM images with virtio-gpu support to run on kvm. Would have GPU acceleration in that case.

How can I go about creating such an image.? Any ideas and hints would be helpful…

I personally have stayed away from anything armhf on arm64. It is reasonable that a QEMU VM could deal with this, but in practice this is probably more difficult than it sounds. I don’t develop under QEMU, so I can’t be of much use with that approach. Add to this that kvm might also be difficult, requiring kernel changes, so this is something out of my range of expertise. There are other people here though who have worked with kvm and QEMU on Jetsons, so they might be able to comment.

Actually I could run Raspbian OS 32 bit on QEMU (but without KVM because I could not use virt as the machine).
https://github.com/dhruvvyas90/qemu-rpi-kernel
KVM patches are available for jetson. I have it enabled. I am even able to run Win10 with KVM.
See this thread: Xubuntu 20.04 Focal Fossa L4T R32.3.1 - Custom Image for the Jetson Nano - #111 by rreddy78

What we need is a Raspbian OS 32 bit with kernel support for virtio drivers and virt machine…

So yes. Its possible to run Linux armhf OS with QEMU+KVM and GPU acceleration!
Here is the screenshot of chromium running on Debian Buster Xfce.

Detailed instructions to follow…

Pre-requisites are:

  1. Kernel with KVM → Covered in other threads
  2. Updated libSDl2 built from sources

Due to a bug in the libSDL2 library available in 18.04, we need an upgraded version to be built from sources.
Download the libSDL2 sources of 19.04 from here:

https://launchpad.net/ubuntu/+source/libsdl2/2.0.9+dfsg1-1ubuntu1.19.04.1

Download the file:
libsdl2_2.0.9+dfsg1.orig.tar.xz

Extract and build:
sudo apt-get build-dep libsdl2-2.0-0
./configure
make
sudo make install

  1. Latest qemu built from sources (so as to enable virglrenderer)
    Ensure you have all dependencies of the qemu configuration installed including the libvirglrenderer0 and libvirglrenderer-dev

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

./configure --target-list=aarch64-softmmu --enable-kvm --enable-sdl --enable-gtk --enable-virglrenderer --enable-opengl --enable-tools

Now the main hints I got from this post:

https://translatedcode.wordpress.com/2016/11/03/installing-debian-on-qemus-32-bit-arm-virt-board/

In that post, kvm is not used and the version of debian used is “jessie”. I updated it to “buster” and used kvm. Here is the installation command for a command line install and console only installation (don’t select any desktop environment during install)

Non-GUI Install
…/qemu/build/qemu-system-aarch64 -M virt -m 2048 -smp 2 -cpu host,aarch64=off -enable-kvm
-kernel installer-vmlinuz
-initrd installer-initrd.gz
-drive if=none,file=hda.qcow2,format=qcow2,id=hd
-device virtio-blk-device,drive=hd
-netdev user,id=mynet
-device virtio-net-device,netdev=mynet
-nographic -no-reboot

Now an easy way to extract the kernel and initrd is given here:
https://gist.github.com/shamil/62935d9b456a6f9877b5

use the partx command to mount individual partitions

To run the VM its the same command line except that the kernel and initrd is replaced by vmlinuz-4.19.0-14-armmp-lpae and initrd.img-4.19.0-14-armmp-lpae which are copied and additional qemu parameter of -append ‘root=/dev/vda2’

GUI-Install (use a 10G qcow2 image):
For GUI we need to use the EFI EDK2 BIOS which is provided in the qemu sources.
Look in foder pc-bios for the file edk2-arm-code.fd.bz2. Extract the edk2-arm-code.fd from it.

Command line for installation is as follows:

The -serial stdio is optional really (only to get and additional serial terminal)

…/…/qemu/build/qemu-system-aarch64 -M virt -m 2048 -smp 2 -cpu host,aarch64=off -enable-kvm
-kernel installer-vmlinuz -initrd installer-initrd.gz
-device usb-ehci -device usb-kbd -device usb-mouse -usb -serial stdio
-device virtio-gpu-pci,virgl=on,xres=1600,yres=900 -display sdl,gl=on
-drive if=none,file=hda2.qcow2,format=qcow2,id=hd
-device virtio-blk-device,drive=hd -netdev user,id=mynet
-device virtio-net-device,netdev=mynet
-bios edk2-arm-code.fd -no-reboot

Install an Xfce desktop for minimal resources.

Again running the changes required are as shown below

-kernel vmlinuz-4.19.0-14-armmp-lpae -initrd initrd.img-4.19.0-14-armmp-lpae -append ‘root=/dev/vda2’ \

this procedure does not work :

https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/libsdl2/2.0.9+dfsg1-1ubuntu1.19.04.1/libsdl2_2.0.9+dfsg1-1ubuntu1.19.04.1.debian.tar.xz

Extract and build:
sudo apt-get build-dep libsdl2-2.0-0

this is what happens :

root@ziomario-desktop:# cd libsdl2_2.0.9+dfsg1-1ubuntu1.19.04.1

root@ziomario-desktop:# ls
changelog gbp.conf libsdl2-doc.doc-base sdl2-config.1
compat libsdl2-2.0-0.install libsdl2-doc.examples source
control libsdl2-2.0-0-udeb.install libsdl2-doc.install tests
copyright libsdl2-dev.install patches upstream
docs libsdl2-dev.manpages rules watch

root@ziomario-desktop:# sudo apt-get build-dep libsdl2-2.0.0
Reading package lists… Done
E: Unable to find a source package for libsdl2-2.0.0

root@ziomario-desktop:# ./configure
bash: ./configure: No such file or directory

Updated the links. Copy paste error

nope. there is a wrong command on your tutorial. but u test the commands before to write them here ? its not good to copy and paste without to try by yourself before.

I have extracted the content from this file :

libsdl2_2.0.9+dfsg1.orig.tar.xz

and then,as u said : Extract and build:

root@ziomario-desktop:# cd /root/Desktop/Work/I9/Anbox/SDL2-2.0.9/

root@ziomario-desktop:# ls
acinclude include sdl2-config.in
autogen.sh INSTALL.txt sdl2.m4
BUGS.txt Makefile.in sdl2.pc.in
build-scripts Makefile.minimal SDL2.spec
cmake Makefile.os2 SDL2.spec.in
CMakeLists.txt Makefile.pandora src
cmake_uninstall.cmake.in Makefile.psp test
configure Makefile.wiz TODO.txt
configure.in README-SDL.txt VisualC.html
COPYING.txt README.txt wayland-protocols
CREDITS.txt SDL2Config.cmake WhatsNew.txt
docs sdl2-config.cmake.in

root@ziomario-desktop:# sudo apt-get build-dep libsdl2-2.0-0
Reading package lists… Done
Picking ‘libsdl2’ as source package instead of ‘libsdl2-2.0-0’
E: Unable to find a source package for libsdl2-2.0-0

No idea. The steps are correct. Only the download link was wrong.
By building a package, it means:

  1. Make sure all build deps are installed. You command to install them is correct.
    Install additional depending on need
  2. Run ./configure in build dir with desired options
  3. make -j4
  4. sudo make install (installs to /usr/local)

sudo apt-get build-dep libsdl2-2.0-0
[sudo] password for rreddy78:
Reading package lists… Done
Picking ‘libsdl2’ as source package instead of ‘libsdl2-2.0-0’
Reading package lists… Done
Building dependency tree
Reading state information… Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

and

rreddy78@jetson-nano:~/Downloads/libsdl2_2.0.9+dfsg1.orig/SDL2-2.0.9$ ls
acinclude cmake configure gen Makefile.in Makefile.rules sdl2-config.cmake sdl2.pc test
autogen.sh CMakeLists.txt configure.in include Makefile.minimal Makefile.wiz SDL2Config.cmake sdl2.pc.in TODO.txt
BUGS.txt cmake_uninstall.cmake.in COPYING.txt INSTALL.txt Makefile.os2 README-SDL.txt sdl2-config.cmake.in SDL2.spec VisualC.html
build config.log CREDITS.txt libtool Makefile.pandora README.txt sdl2-config.in SDL2.spec.in wayland-protocols
build-scripts config.status docs Makefile Makefile.psp sdl2-config sdl2.m4 src WhatsNew.txt

rreddy78@jetson-nano:~/Downloads/libsdl2_2.0.9+dfsg1.orig/SDL2-2.0.9$ ./configure
checking build system type… aarch64-unknown-linux-gnu
checking host system type… aarch64-unknown-linux-gnu
checking how to print strings… printf
checking for gcc… gcc
checking whether the C compiler works… yes
checking for C compiler default

works perfectly for me…

this website is very useful to learn how to extract files from a qcow2 image.

BTW in this tutorial you have been very clear and exhaustive. thanks.

With this additional xserver config file on nano and installed nvidia-driver package in armhf chroot ( Itested bullseye), It’s possible to run some apps and games with indirect rendering. Box86 crashes so i386 emulation is still a bit no-go, unless you want to use virpipe and virglrenderer. 10-monitor (3).conf (402 Bytes)