Why all the errors on bootup

How is your imx219 connected? Perhaps the system is looking for more than one camera. In the case of plug-n-play devices, e.g., USB, detection and finding a driver to associate is more or less automatic. In the case of a non-plug-n-play method, the device tree is used to find the hardware, and is blindly trusted to be correct. So it is possible that if there is a device tree setting for one camera which is not there, and another camera is being set up which is there.

Note that the i2c control for a camera, despite being for the same camera as some other setup, is really a separate setup. Most i2c is described via device tree, but I suppose there are cases of i2c over USB in which it is automatic.

Some devices do use firmware. Firmware is either a method of finding hardware (such as the device tree, which loads into the Jetson), or else is uploaded into the device. Wi-Fi devices very very often have firmware to upload into them. The laws in different parts of the world regarding radio transmitter devices (which Wi-Fi is one such device) differ depending where you are; one could build different hardware for each region of the world, or one could stop selling a product anywhere except particular parts of the world, or better yet, one could load software into the device itself which defines the device behavior, and use the same hardware in every part of the world simply by coupling it with the correct firmware.

I see there is an attempt to upload firmware into the iwlwifi device, and that this is failing. When firmware is uploaded into the device, it tends to be independent of the host architecture. This is because it isn’t being run by the host, it is simply uploading from the host, and it is the device itself using that firmware. I am thinking that for the Wi-Fi the manufacturer has firmware which needs to be present in the correct subdirectory of “/lib/firmware”, but that it is missing (or perhaps in the wrong place). Many devices which use firmware have a default behavior when no firmware is uploaded, and other devices simply won’t function correctly without the firmware (some devices, with the wrong version of firmware, will bring down the system).

A device tree can also be wrong regarding what driver and/or firmware it is working with and cause similar error messages. Overall, I don’t know if what you are seeing is from missing firmware, or bad device tree settings, or some combination. Most likely firmware is missing, but you wouldn’t be able to debug all of the issues at once, you’d need to start with the firmware and figure out if the imx219 needs firmware, followed by seeing if it is in “/lib/firmware”, and also by noting if this is connected as a plug-n-play device versus device tree.

The only setup i have done for the csi camera is via sudo /opt/nvidia/jetson-io/jetson-io.py option and setting for imx dual camera
Will check the lib/firmware folder as mentioned.
In case i have messed up something is there a sort of master reset that puts everything back to factory and start from scratch if needed.

I have looked in the lib/firmware folder and there is a lot of the iwlwifi device .ucode files but i am not sure what i am looking for the imx219 camera. Added a listing of the folder.
files.txt (4.9 KB)

I see this subset of files, apparently listed in alphabetical order:

iwlwifi-8265-21.ucode
iwlwifi-8265-22.ucode
iwlwifi-8265-27.ucode
iwlwifi-8265-31.ucode
iwlwifi-8265-34.ucode
iwlwifi-8265-36.ucode

I see these complaints of missing firmware:

[    4.389480] iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-26.ucode failed with error -2
[    4.398793] iwlwifi 0000:01:00.0: Falling back to user helper
...
[    4.886948] iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-25.ucode failed with error -2
[    4.897502] iwlwifi 0000:01:00.0: Falling back to user helper
[    4.905055] iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-24.ucode failed with error -2
[    4.914367] iwlwifi 0000:01:00.0: Falling back to user helper
[    4.920885] iwlwifi 0000:01:00.0: Direct firmware load for iwlwifi-8265-23.ucode failed with error -2

Looks like there might be more files that are needed in your “/lib/firmware”:

iwlwifi-8265-23.ucode
iwlwifi-8265-24.ucode
iwlwifi-8265-25.ucode
iwlwifi-8265-26.ucode

I do not know who or what provides that firmware. However, since this is uploaded into the camera, then likely the same file uploaded by a PC or any other operating system, and that same file will work here as well…it should be operating system and architecture agnostic (the architecture is in the sensor itself, not the host computer). This means that if you find that file for any operating system or hardware, that you can probably just copy it to the right location on the Jetson and it would “just work”. Don’t know for sure. Maybe someone else knows where to get the iwlwifi-8265 firmware.

Ok well that gives me some direction to chase after, thank you very much for your help.

I know everything on my intel ac8265 wireless board is working including wifi and bluetooth so was curious to just try and supress the errors listed. The file iwlwifi-8265-22.ucode is shown as the correct file and the only one available on the intel site. Anyway i made copies of the iwlwifi-8265-22.ucode and renamed to the missing files and moved them to the lib/firmware folder. After rebooting the errors were supressed. Everything on wifi and bluetooth still seems to work correctly so until the correct fix can be found this will have to do. Thanks again for your help.

Fascinating test! Glad it worked. One other possibility, if those files which do exist are from a package, is to find out what package provided those. For example, since you have this file, check if this shows a package:
dpkg -S /lib/firmware/iwlwifi-8265-22.ucode
(I’m assuming no subdirectory, but adjust that for whatever subdirectory the actual file might be in)

No there is no sub directory and the only result from the dpkg command is:
linux-firmware: /lib/firmware/iwlwifi-8265-22.ucode

When i run the dpkg command with iwlwifi* its list all with similar linux-firmware: except the ones i renamed it says “dpkg-query: no path found for matching pattern /lib/firmware/iwlwifi-8265-23.ucode” etc for the others too.

Ignoring the lib/firmware directories this is the found wildcard files for iwlwifi found on the system:

etc/modprobe.d/iwlwifi.conf
lib/modules/4.9.337-tegra/kernel/drivers/net/wireless/intel/iwlwifi
lib/modules/4.9.337-tegra/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
proc/irq/408/iwlwifi
run/udev/data/+module:iwlwifi
run/udev/data/+drivers:pci:iwlwifi
sys/kernel/debug/iwlwifi
sys/kernel/debug/ieee80211/phy0/iwlwifi
sys/bus/pci/drivers/iwlwifi
sys/module/iwlwifi
sys/module/iwlwifi/drivers/pci:iwlwifi
sys/module/cfg80211/holders/iwlwifi
usr/src/linux-headers-4.9.337-tegra-ubuntu18.04_aarch64/kernel-4.9/include/config/iwlwifi.h
usr/src/linux-headers-4.9.337-tegra-ubuntu18.04_aarch64/kernel-4.9/include/config/iwlwifi
usr/src/linux-headers-4.9.337-tegra-ubuntu18.04_aarch64/kernel-4.9/drivers/net/wireless/intel/iwlwifi
usr/share/apport/iwlwifi_error_dump
usr/share/doc/linux-firmware/licenses/LICENCE.iwlwifi_firmware

The lack of a package implies the install was not from one of the Ubuntu servers. For example, it was probably part of a camera tarball that was unpacked without the package tool, and hints that the original file came from the hardware vendor. This is of course not definitive, but could be considered “typical” of Wi-Fi devices.

Content in “/lib/modules” is from the kernel build and install. The same is true of “/sys” files. I think content in “/var/run” might be part of a runtime detection, e.g., hot plug or auto mount. The one part which might really be useful is to check with “dpkg -S ...” against the files you saw in “/usr/share” (including “doc/”): Those could have a package which installed them, especially the license file. In fact, I would read the LICENSE.iwlwifi_firmware as this could mention the source of those files. If there is a URL, then you might check out what is there in a web browser.

1 Like

Can i ask a question, i am starting to wonder that although i bought the Nano 4GB B01 as brand new i am wondering if it has been used before. I am confused as why i used a fresh install of jetson-nano-jp461-sd-card-image.zip on a new card and with nothing connected to the Nano, so no imx219, no iwlwifi wireless m2 card etc, yet on this new sd card image boot i still get the imx and iwlwifi errors as shown in the #1 post. Does the Nano hold previously installed hardware settings even if you write a new image and disconnect everything. Sorry for the lack of knowledge on this.

Jetsons do not have a BIOS. As such, the duties of a BIOS are performed in software. For SD card models, this exists in QSPI memory on the module itself. On eMMC models, this is all in eMMC partitions. The gist though is that a full flash (and an SD card is only a single partition, it is not full flash) essentially is a flash of:

  • BIOS (it’s equivalent)
  • Bootloader.
  • Operating system (the “APP” partition label).

There are patches over time for the QSPI content, and if the SD card gets a newer release, then “often” the QSPI itself must be updated for the SD card to be useful. The modules do have old QSPI content; you could “sort of” say that this means they have an out of date BIOS (though it isn’t really a BIOS). It would be correct to say that there is an old BIOS installed, but the hardware is new. The newer QSPI would not have existed at the time of manufacture of the hardware.

Sometimes boot stages (before the Linux kernel loads) will be required to have drivers or firmware as well. Bootloaders are really an operating system, but their only job is to eventually overwrite themselves with the next stage (the Linux kernel). Some drivers also require firmware, e.g., often Wi-Fi. Not all hardware needs to run in boot stages, but if it does try to run there, or if some sort of device setup is needed, then this could include a need for firmware (and if the QSPI memory is too out of date, then the update would also include firmware update as part of the QSPI flash). I don’t know for your specific case, but it is very common for newer SD cards to not work with older QSPI content which is installed at the factory. It is random luck whether you get updated QSPI or not from a new unit. It is always advised to flash an SD card model at least once with the newest release after a purchase.

Incidentally, commercial modules (which would have eMMC) are shipped completely empty. It’s only the dev kits which have any content on them.

Considering every piece of additional hardware is working fine its started to seem i am going to have to live with it, especially as my Nano b01 is discontinued now.

Before i commit to giving up and living with the annoying errors listed can you confirm that what you mentioned is what this chap is doing with firmware from the https://auvidea.eu/firmware/ site and shown achieving this on https://www.youtube.com/watch?v=id2UDqA4X-4 Thanks

The first part of that video shows he is flashing with just stock JetPack/SDK Manager, but he has to solder a header on to get the 0.1" spacing recovery pin exposed with that custom carrier board (once the Jetson has been powered up with the recovery pin shorted to ground you can remove the header; you don’t need to leave it in place, and in fact there are advantages to removing the pin short right after powering up). It is in the second part that he is using the Auvideo software, which is where the device tree (firmware) is different.

Note that when he unpacks the .tar.gz content which is specific to the Auvidea board that it produces a “bootloader/” subdirectory. This is where it becomes customized. He is using (within that) what is probably “mostly” from the NVIDIA flash content, but modified to use a custom device tree. This is where it would “fix” a lot of error messages.

Note that stock NVIDIA flash software produces this location when installed via JetPack/SDK Manager:
~/nvidia/nvidia_sdk/JetPack...version.../Linux_for_Tegra/

That location also has subdirectory “bootloader/”. My guess is that most of what is provided is exactly the same between the two sets of flash software except for the firmware which is in bootloader/, and the flash command for command line flashing of that specific board.

The part where he is soldering the pins , i don’t think i need to do that on mine do it as i thought it would be using pins 9 & 10 on the B01 developer kit. Is that correct.?

The guy in the video isn’t using a developer kit, and developer kits do have those pins. The second part is also incorrect if and only if this is not developer’s kit (the device tree and other software of a developer kit don’t use custom software). You have to distinguish between a module on a developer’s kit carrier board, versus a module on a custom or third party carrier board; it is the carrier board which determines the custom device tree requirements.

Wi-Fi can have more customization requirements on any carrier board. Different governments around the world regulate Wi-Fi differently. There is a subset of radio frequencies/bands which every region seems to accept, and there are others which are available in some places, but not in others. A manufacturer could make any of these choices for Wi-Fi:

  • Produce a Wi-Fi device which works only on the shared subset of radio frequencies.
  • Produce different Wi-Fi hardware for each region of the world.
  • Stop selling in different regions of the world.
  • Control the radio bands and transmitter strengths based on firmware uploaded into the Wi-Fi such that the hardware becomes compliant with the region it will be shipped to.

NVIDIA takes the first approach in cases where it integrates Wi-Fi into its hardware. In cases where there is add-on Wi-Fi, e.g., via an m.2 connector or USB, it becomes the Wi-Fi manufacturer’s choice. USB and m.2 are plug-n-play devices, and if the firmware exists, and if the manufacturer has installed the software to trigger upload of the correct firmware into the Wi-Fi hardware, then it “just works”.

There is also non-plug-n-play hardware. As an example, the USB controller itself exists at some physical address, and the drivers must be told where to find that controller, and which driver is compatible with that chipset. Once this is done, the SoC itself has a number of pins which can be routed to different hardware based on device tree, and that USB controller might have several choices of which pin routes to the copper traces the carrier board actually uses. This gives each manufacturer a chance to customize layout in ways that would not exist if every wire were hard-wired. In that case the firmware (device tree) depends on the carrier board.

In the case of an m.2 slot there would be a device tree for finding the wiring going to that slot. There might also be firmware which uploads into any device connected to the m.2 slot. Any Wi-Fi chipset might have yet another set of firmware.

In the case of a plug-n-play Wi-Fi chipset the flash of the Jetson itself won’t be related to the Wi-Fi chipset firmware. Installing a “driver” for such a Wi-Fi chipset might really be (A) an actual hardware driver, plus (B) firmware such that the hardware behaves correctly with that driver. Mismatching uploaded firmware for Wi-Fi (or not uploading firmware when it is needed) can have unpredictable failures (anywhere from non-function to bad behavior or complete system crash).

If you truly have a developer’s kit, then you have module without eMMC, but it will have QSPI. Flash of the QSPI (the module itself) using the stock board support package (BSP; the flash software and content that gets flashed) guarantees that the module and device tree for this is correct, and that lanes are routed correctly using the right pins of the module to the correct lanes of the carrier board. In the case of a Jetson which comes with Wi-Fi hardware, it also implies you’d have the right software for that hardware (which might or might not include firmware which uploads into the Wi-Fi).

I don’t know about your specific warnings. Some of them should not occur if flashed with the correct release of software on a dev kit. If you use an Auvidea carrier board, then you are not using a dev kit, and all of the above “correctness” may no longer be correct. It then depends on whether the Auvidea board is an exact electrical match in layout to the dev kit. Sometimes manufacturers use an exact match, but more often they do not clone the electrical layout of the dev kit. In this latter case, the second half of that video is correct because it uses their custom BSP. Flashing a non-matching system with the dev kit software will lead to errors, warnings, and parts of the hardware not functioning. I just don’t know what is in your carrier board, but if it is Auvidea, then you use their BSP; if it is truly a dev kit, then you use the NVIDIA software for flash.

Thanks for the lesson, its starting to register in my head now.
While i was waiting for a reply i knocked up a Pc with ubuntu 18.0.4 and ran the SDK Manager 2.0 connected to it with it in FC mode , everything ran fine in regards to flashing but still the old imx219 i2c warning on the bootup screen. As mentioned before though everything works fine in regards the hardware installed including Camera, WIFI & Bluetooth.
This is the exact kit i bought 4GB Nano Dev Kit so i have been assuming everything is original stock and not customised.

It does say it is a dev kit, and the picture looks like a dev kit, so if the quick start guide says to use the NVIDIA software, then I would say it really is a dev kit. Being a 4 GB module, the compatible releases would be somewhat fewer, but all of the JetPack 4.x/L4T R32.x more recent releases would be compatible (older releases would not be compatible). These URLs would say which releases are compatible (one if looking up via JetPack/SDKM, the other if looking up L4T version):

There’s a question for NVIDIA: Are the same releases shown for TX2 4GB valid for Nano 4GB? If not, then Nano 4GB probably needs to be added to the L4T release version listings.

One thought is that if you have an imx219, and it works, that the i2c could be a separate issue. If you have hardware which can be controlled via i2c, and if that hardware functions and is controllable, then it is a different i2c issue. If there is i2c circuitry for controlling the sensor, and it does not work, then the sensor must be working with defaults and the i2c would have to be fixed for that control to work. I don’t know which i2c it is. If you have some software to control aspects of the camera, then I’d suggest testing those aspects to see if they work or fail (and to indirectly find out if the i2c issue matters; perhaps it is for a second camera, in which case it doesn’t matter).

Yes everything on the I2c works from the camera aspect and the pca9685 servo controller. I have used it no problem with object detection, face tracking with servos and all gst-launch-1.0 nvarguscamerasrc queries return all specs fine. The one thing that previously made me think was as you mentioned was when using the jetson-io configuration the configurable hardware selection only gives (imx219 dual camera) or (imx219 and imx477) option for the i2c pins setup so is it looking for a second camera. Maybe i should grab another camera and populate both camera slots to see if that makes a difference.

If you have a second camera, then that sounds like a good idea.

1 Like