Accessing CoreSight Program Trace Macrocell (PTM)

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello everyone,

I am trying to access the CoreSight Program Trace Macrocell (PTM) as described here on the Jetson Nano. It runs L4T R32.7.1.

However, when checking if CoreSight is configured in the kernel, it says that “CONFIG_CORESIGHT is not set”.

So, I want to configure the kernel and follow the steps described here.
I am new to building Linux kernels and have some questions:

  1. Is it the right approach to include CoreSight in the kernel configuration via make menuconfig? The documentation mentioned above says nothing about actually configuring the kernel, only compiling it.
  2. Is there a way to verify that the freshly compiled kernel does have CoreSight configured?
  3. How do I proceed after compiling the kernel? What do I have to do to boot the Jetson Nano with the new kernel and finally being able to use CoreSight? The documentation says nothing about that too.

Any help would be highly appreciated!

Best regards,
Vincent

3

Hi,
Please follow the steps to rebuild/replace kernel image:

https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/kernel_custom.html#wwpID0E0XD0HA

You can run the command to ensure the configs are enabled:

$ zcat /proc/config.gz

4

Hi,
like I said in my post, I already knew that I had to follow the steps of building the kernel. However, my questions go beyond that, as the instructions only describe like half of the part of actually configuring the kernel.

Please answer my questions directly and not just paste information that I already included in my post. Thank you in advance!

5

I have not set up CoreSight, but the general theme is that you start by configuring your kernel to be an exact match to the running kernel. That “normally” also means setting CONFIG_LOCALVERSION to match.

If you are going to change anything, and add CoreSight such that only modules are added, it is a case of not changing CONFIG_LOCALVERSION. If you are going to change the core kernel Image (and “=y” feature change rather than an “=m” change), then you must rebuild everything, and the CONFIG_LOCALVERSION must change. You still start by otherwise matching the running kernel (but now CONFIG_LOCALVERSION will no longer be “-tegra”).

Once your base config matches, and you have picked the correct CONFIG_LOCALVERSION (though this can be changed after if you start out assuming no =y change), then you can use tools like menuconfig or nconfig (I like this latter, it has symbol search) to make your changes for CoreSight.

The “/proc/config.gz” is just a reflection of the current kernel’s running config (this does not include CONFIG_LOCALVERSION, but otherwise it is an exact mirror of the running kernel’s config). If you have a stock kernel, then config.gz can be copied, decompressed, and renamed to .config in your build location as a method to set the base config (then you’d go straight to menuconfig or nconfig). Or you can use that file to verify after booting a new kernel to see if non-module features were updated (you’d use lsmod to see module; only the modules at the original build time show up in config.gz…later additions won’t show).

6

I want to change the core kernel Image (set CoreSight to “=y” like in the link above), no modules.
What do you mean with matching to the running kernel? And where can I find CONFIG_LOCALVERSION?

Is it not enough to run make menuconfig after step 3 of these instructions and make sure that “CONFIG_CORESIGHT=y” etc. is set in .config before compiling the kernel? It says that the CoreSight files in /drivers/hwtracing/coresight got compiled during kernel compilation.

However, my biggest problem still remains: What do I have to do after step 8? How do I boot the new, compiled kernel on the Jetson Nano so that I can finally use CoreSight?? (I am compiling directly on the Nano, no cross-compilation.)

7

Some basic trivia on kernels and their configuration might be of use.

Both modules (set with “=m”) and integrated features (set with “=y”) are part of the kernel. However, modules can be added as the kernel runs. If you have an existing kernel, which is what I’ll call only the “=y” features, then it is easy enough to build a module which can plug in to that integrated feature set. Technically, in the configuration file, those are called symbols (it is part of the “kbuild” system).

If you plug in anything dynamically, then there is both an API and an ABI (application binary interface) which must be followed. If you are set to a given set of integrated features (“=y” symbols), on the same source code, then you could say that the “shape” of the plugin remains constant and you can expect any module to be able to plug in. As soon as you change an “=y” integrated feature all guarantees are gone so far as module components being interchangeable.

One of the symbols for a Linux kernel is “CONFIG_LOCALVERSION”. When you run the Linux command “uname -r”, this is a reply from the kernel. The prefix of the output is the kernel source version, and the suffix is the CONFIG_LOCALVERSION. If you run “uname -r”, and get the result “5.15.0-tegra”, then the “5.15.0” is the source version of the integrated kernel, and the “-tegra” is the setting of CONFIG_LOCALVERSION at the time of compile:
CONFIG_LOCALVERSION="-tegra"

This, in turn, is how a kernel can find modules to load. This is also how a single kernel version can have multiple boot entries if desired. If you run a kernel which replies to “uname -r” as “5.15.0-tegra”, then it will find its modules here:

  • /lib/modules/$(uname -r)/kernel
  • …which is “/lib/modules/5.15.0-tegra/kernel”

Now if you’ve changed an integrated feature symbol, and you keep the “uname -r” the same via constant “CONFIG_LOCALVERSION”, then you might find your system won’t boot (or at least the modules for the original kernel will possibly fail). If you have a backup of the original kernel, and instead of the new kernel having the same “CONFIG_LOCALVERSION”, for example “CONFIG_LOCALVERSION=-debug”, then the original kernel can remain a working backup, and the new kernel will look here for its modules:
/lib/modules/5.15.0-debug/kernel
…and you can pick boot entries and choose either.

Because you are altering an integrated symbol which is invasive, it is almost certain the old modules won’t work correctly. You need to build with all configuration matching, except for your updates and maybe “CONFIG_LOCALVERSION=-coresight” (notice I do have a hyphen). Then you might consider saving your original kernel Image file and not altering it. Instead, add your new kernel as:
/boot/Image-coresight

Then install 100% of all kernel modules. These will end up at “/lib/modules/$(uname -r)/kernel” (it is part of the modules_install target).

If you then add a secondary boot entry which points at the -coresight kernel, you get that in boot. If you still have an entry to the original Image, then you also have that.

The above is just about kernels, and I can’t tell you the requirements for CoreSight. However, if you want to add the requirements in kernel features, then that’s where you start. Find the configuration of the default running kernel, configure your source to that. Make your modifications to the “=y” of symbols. Set CONFIG_LOCALVERSION to something like “-coresight”. Install Image as Image-coresight. Install all modules (they will end up in the -coresight subdirectory for modules). Set a new boot entry while leaving the original for rescue.

A huge complication: If you are using an initrd boot, e.g., with external media, then kernel install itself changes somewhat and you must work with the different procedures for that. The integrated kernel Image is still on a /boot somewhere, but any modules which are mandatory to loading the kernel and the filesystem have to exist in the initrd. Remember what I said about changing an integrated symbol probably causing old modules to invalidate load? It is possible for an initrd case that you would also have to put modules in a new initrd. The initrd itself can be named in a boot entry paired with the new kernel, and you only need this for boot stage, but it is possible adding the kernel also requires a second initrd.

EDIT: Forgot to mention, that step is just showing you how to tar a full new set of modules you’ve just built, copy it over to the Jetson, and untar it. Do a test run in an empty directory, see if it shows “/lib/modules/$(uname -r)”, where the “uname -r” is from the new kernel version and CONFIG_LOCALVERSION.

8

First of all, thanks a lot for the detailed explanation!

However, I looked into /proc/config.gz and found that CONFIG_LOCALVERSION is only set to an empty String and not to “-tegra”. Is that a problem when I set it to “-coresight” for my configuration?

You need to build with all configuration matching, except for your updates and maybe “CONFIG_LOCALVERSION=-coresight”

Just to check if I understood this sentence correctly: I still configure the .config file via make menuconfig (and set the CoreSight CONFIGs to “=y” as well as CONFIG_LOCALVERSION=“-coresight”), but have to make sure that everything else is exactly the same as in /proc/config.gz? If not, what do you mean with “build with all configuration matching”?

9

I should emphasize that config.gz is an exact copy of the running kernel except it never sets CONFIG_LOCALVERSION. You must extract that by finding the suffix from the command “uname -r”. Then you must set it yourself (either to match if keeping all “=y” features the same and not altering them, or something new if changing “=y” features).

Anything that matches your current config to start with is the valid first step. If you have an unmodified kernel, then the tegra_defconfig target will do that. In every case the config.gz copied in, gunzip’d, and renamed as .config will work even if modified. In all cases you must then set CONFIG_LOCALVERSION. Then you can use the mentioned make menuconfig as you are using your example.

By the way, “make nconfig” is very much like “menuconfig”, but you’ll see a symbol search function. You don’t need to use the CONFIG_ to search for symbols, but you could for example search for “CONFIG_LOCALVERSION” or “LOCALVERSION” if you are setting this with an editor (I advise all settings through such a dependency-aware editor, but I happen to know that CONFIG_LOCALVERSION can be edited directly in the .config file since it has no dependencies).

A more detailed explanation is that when you build or configure you are setting or editing a .config file. If the source is somewhere separate from the output, then the output location is where that file is. If the source is the same as compile location, then it is at the root of the source. Commands like “make tegra_defconfig” will create such a file if not already present. Commands like “make nconfig” will edit the file which already exists; if none exists it will create a broken version of this. I will emphasize that none of the commands will set up CONFIG_LOCALVERSION, you have to do that yourself.

Shorter explanation: Regardless of whether you are working with an unmodified kernel or a modified kernel, if that kernel works and is where you want to start, then you can create your .config via a copy, gunzip, and rename of the /proc/config.gz (you will always have to set the CONFIG_LOCALVERSION though).

10

Thanks to your help, I finally managed to compile the kernel and boot into it (it seems that I missed copying the modules to /lib/modules/ in my previous attempts).

Unlike before, /proc/config.gz shows that CoreSight is now configured, and the coresight folder under /sys/bus/ exists as well.

However, /sys/bus/coresight/devices is empty and I still can’t use it. Does this have to do something with the Device tree? I only changed Image and extlinux/extlinux.conf in /boot/ before rebooting into the new kernel. I feel like I have to do something with the files in $TEGRA_KERNEL_OUT/arch/arm64/boot/dts/ like they said in step 7 of the kernel customization documentation.

11

I don’t know what CoreSight wants. I can tell you though that the existence in /sys and in /boot/config.gz says that (A) the driver is present, and (B), the driver is running. I’m only guessing, but there is probably some other requirement to use it. A common example for software similar to this is an argument being passed to the kernel as it loads to tell it to run with that feature. For Jetsons that argument would become part of the long “APPEND” line (space delimited) in “/boot/extlinux/extlinux.conf”. Do the CoreSight docs mention anything like this?

Also, if this is used with KGDBOC to trigger, then you’d need to run the correct echo to a file in /proc to trigger the feature. I don’t know if CoreSight has anything to do with KGDBOC, but that is an example method of triggering for KGDB software kernel debug (and this should be similar to CoreSight). But, I don’t really know, you’ll have to dig into documents. Any mention of CoreSight in a serial console boot log (with “quiet” removed) might offer hints.