I am trying to apply xenomai on Jetson AGX orin, i patched the Jetson Linux with the dovetail and xenomai（Jetson Linux: 35.1 with Linux kernel version 5.10.104; Dovetail: 5.10.127-dovetail patch; Xenomai: xenomai stable v3.2.x), then built and flashed the Image & KO file into the orin device. When i reboot the orin device, the kernel stopped while running do_initcalls(), and i got the log data through the serial port as the attachment.
Anyone knows how to fix this problem? Any help will be appreciated~
orin_xenoDove_image&ko_boot_log_serialport.txt (103.8 KB)
Hi,
how did you confirm where the booting process got stuck?
There’s no do_initcalls() in the booting log.
Thank you for your reply,
in my debug version, i add some pr_info() statements in the do_initcall_level() of kernel code as the following,
static void __init do_initcall_level(int level, char *command_line)
{
…
pr_info(“—loginfo–level=%d,total-count=%ld\n”, level, (initcall_levels[level+1] - initcall_levels[level]));
for (fn = initcall_levels[level]; fn < initcall_levels[level+1]; fn++) {
pr_info("---loginfo--level=%d,count=%ld\n", level, (fn - initcall_levels[level]));
do_one_initcall(initcall_from_entry(fn));
}
}
in the booting log of this kernel , the last block contains the keyword “loginfo” is like this:
[ 16.614585] —loginfo–level=7,total-count=63
…
[ 16.942830] —loginfo–level=7,count=33
[ 16.949795] tegra-soc-hwpm f100000.tegra_soc_hwpm: Adding to iommu group 13
[ 16.960174] host1x 13e40000.host1x: Adding to iommu group 14
[ 17.976896] t23x_init_gating_regs: Timed out waiting for syncpt ram init!
[ 17.994115] host1x 13e40000.host1x: initialized
[ 17.999203] iommu_context_dev 13e40000.host1x:niso0_ctx0: Adding to iommu group 15
obviously, some init functions of level 7 not run.
i have checked the log data in other levels of initcall process, the value of “total-count” matches the last value of “count” in each level initcall process.
also, the native orin kernel image has some other log information of power:
[ 14.354758] vdd-dp: supplied by vdd-3v3-sys
[ 14.359257] vdd-3v3-sd: supplied by vdd-3v3-sys
[ 14.364061] dsi-vdd-1v8-bl-en: supplied by vdd-1v8-sys
[ 14.370970] irq: IRQ319: trimming hierarchy from :pmc@c360000
[ 14.376963] irq: IRQ320: trimming hierarchy from :pmc@c360000
so i think my xenomai kernel stopped/blocked in the “initcall” procedure.
Hi,
Did you find any examples doing this on Jetson? Not sure if it’s something we can handle.
Can you switch to the PREEMPT_RT kernel if you need a real-time environment, and maybe also try the latest r35.3.1?
thanks for your suggestion, we want to add some features into xenomai and implenent it on some device, not just let our device be real-time, so i have to go forward with xenomai.
i got some other information today i think may be useful.
when I reserved the device tree files in “Linux_for_Tegra/kernel/dtb/” rather than replace them with the files in “kernel_out/arch/arm64/boot/dts/nvidia” (the Image and ko files are replaced with the compiled files as before), the boot problem in initcalls mysteriously disappeared, but some other problems about module_layout version emerged, also the monitor did not show any pictures. the last block of boot log is as follows.
[ 18.656756] using random self ethernet address
[ 18.661380] using random host ethernet address
[ 19.238526] using random self ethernet address
[ 19.243345] using random host ethernet address
[ 29.175550] nvidia: disagrees about version of symbol module_layout
[ 29.307632] nvidia: disagrees about version of symbol module_layout
[ 31.925637] Please complete system configuration setup on the serial port provided by Jetson’s USB device mode connection. e.g. /dev/ttyACMx where x can 0, 1, 2 etc.
looks like the system boot process nearly finished, but some ko files were not updated.
i searched all the ko files in “Linux_for_Tegra/”(flash folder not the compile folder) and check the module_layout for each with the command “modprobe --dump-modversions [ko-file] | grep module_layout”, and i actually got the following ko files with different “module_layout”:
Linux_for_Tegra/rootfs/usr/lib/modules/5.10.104-tegra/extra/opensrc-disp/nvidia-drm.ko
Linux_for_Tegra/rootfs/usr/lib/modules/5.10.104-tegra/extra/opensrc-disp/nvidia-modeset.ko
Linux_for_Tegra/rootfs/usr/lib/modules/5.10.104-tegra/extra/opensrc-disp/nvidia.ko
i am not sure if these files are the problem, and i don’t know how to generate the new version of these ko files.
Hi,
glad you had some progress on this issue.
Seems like you also need to build display kernel modules to have all of them work,
can you follow this guide and give it a try?
https://docs.nvidia.com/jetson/archives/r35.2.1/DeveloperGuide/text/SD/Kernel/KernelCustomization.html#to-build-display-kernel-modules
Also, you may use Linux_for_Tegra/tools/l4t_create_default_user.sh to skip the process of creating user accounts and make sure the system can successfully boot up, but just without GUI, which may be solved by what I mention above.
i can enter the graphic interface now after updating the upper 3 ko files, deeply grateful to you for your help~~
there are still some problems about the peripheral devices, for example, the monitor uart channel continuously prompts the warning information:
[ 3572.283518] WARNING: CPU: 0 PID: 0 at drivers/gpio/gpio-tegra186.c:937 tegra186_gpio_irq+0x1ac/0x1f0
[ 3572.293385] —[ end trace d292329a57e2b750 ]—
do you have any suggestions?
Are you using a custom board here?
i think yes, my board is supplied by a third-party company, i think some secondary development has been performed, at least the shape and outline is not as same as the native Jetson AGX orin.
i meet another problem, how can i add my custom command line for kernel boot? the booted system does not contain the /etc/default/grub file.
Hi,
That gpio irq error is not fatal. So could be ignored.
Also one question here. Could you ask just the questions that is regarding to your original topic?
For example, we are okay to reply how to add kernel cmdlines or how to build kernel, because it seems related to your original topics.
But I hope we won’t get every issue all in this topic. If there is something that is not related to original one, please file new topic.
And for kernel cmdline, you can add it in /boot/extlinux/extlinux.conf
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.