Jetson Orin Nano CoreSight ETM enabled in kernel but /sys/bus/coresight/devices is empty

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

Hi NVIDIA team,

I am trying to enable ARM CoreSight ETM tracing on Jetson Orin Nano (T234), but I cannot bring up any CoreSight devices even after rebuilding the kernel with CoreSight options enabled.

Environment

  • Board: Jetson Orin Nano Devkit
  • SoC: T234
  • JetPack/L4T: JetPack 6 / R36.4.3
  • Kernel: 5.15.148-tegra

What I changed

I rebuilt the kernel and enabled:

  • CONFIG_CORESIGHT=y
  • CONFIG_CORESIGHT_LINKS_AND_SINKS=y
  • CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
  • CONFIG_CORESIGHT_SOURCE_ETM4X=y

I replaced kernel image + DTB and rebooted successfully.

Reproduction / Current status

zcat /proc/config.gz | grep CORESIGHT

Output:

CONFIG_CORESIGHT=y
CONFIG_CORESIGHT_LINKS_AND_SINKS=y
CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
CONFIG_CORESIGHT_SINK_TPIU=y
CONFIG_CORESIGHT_SINK_ETBV10=y
CONFIG_CORESIGHT_SOURCE_ETM4X=y

ls -la /sys/bus/coresight/devices

Output:

total 0
drwxr-xr-x 2 root root 0 ...
drwxr-xr-x 4 root root 0 ...

sudo dmesg | grep -i coresight

Output:

[    ...] cs_system_cfg: CoreSight Configuration manager initialised

Also, decompiling the running DTB does not show coresight/etm/tmc/funnel/replicator nodes.

My understanding

This no longer looks like a kernel Kconfig issue.
It seems T234 CoreSight DT/platform description is missing in my current source/release.

I saw forum comments mentioning:
hardware/nvidia/soc/t23x/kernel-dts/tegra234-soc/tegra234-soc-coresight.dtsi
but I cannot find a usable public file/patch for this on my side.

Request

Could you please provide one of the following for R36.4.3 (or the nearest supported release):

  1. A working tegra234-soc-coresight.dtsi; or
  2. A patch set (DTS/DTSI + dependencies) that can be applied directly; or
  3. A clear statement that ETM is not supported on this release, and recommended alternatives.

If supported, please share:

  • ETM/TMC/Funnel/Replicator MMIO addresses
  • required clock/reset bindings
  • ports/endpoints topology
  • expected validation output (/sys/bus/coresight/devices node list)

Thanks in advance.

References

4

Hi,
Please get the file from

Jetson Linux | NVIDIA Developer
Driver Package (BSP) Sources
/hardware/nvidia/soc/t23x/kernel-dts/tegra234-soc/tegra234-soc-coresight.dtsi

It shall be eligible for Jetpack 6. Please give it a try.

5

Thanks for sharing the tegra234-soc-coresight.dtsi path.
I tested it on JetPack 6 (R36.4.3), but ETM/TMC still do not bind successfully.

Environment

  • Board: Jetson Orin Nano Devkit (T234)
  • JetPack/L4T: R36.4.3
  • Kernel: 5.15.148-tegra

What I did

  1. Found file in R35.6.4 public_sources.tbz2:
    hardware/nvidia/soc/t23x/kernel-dts/tegra234-soc/tegra234-soc-coresight.dtsi
  2. Ported this dtsi into my R36.4.3 source tree and included it in board DT chain.
  3. Adjusted obvious compatibility issues for R36 (clock provider/cpu phandle mapping).
  4. Rebuilt DTBs, deployed DTB, updated initrd, rebooted.

Current result

  • Kernel config is enabled:
    • CONFIG_CORESIGHT=y
    • CONFIG_CORESIGHT_LINKS_AND_SINKS=y
    • CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
    • CONFIG_CORESIGHT_SOURCE_ETM4X=y
  • Boot log shows only:
    • cs_system_cfg: CoreSight Configuration manager initialised
  • No ETM/TMC probe success logs.
  • Driver directories show no bound devices:
    • /sys/bus/amba/drivers/coresight-etm4x → only uevent
    • /sys/bus/amba/drivers/coresight-tmc → only uevent

Request

Could you please provide a known-good patch/config for R36.x (preferably R36.4.3 or nearest) including:

  1. Exact tegra234 CoreSight DTS/DTSI content for this release,
  2. Required clock/reset/power/security settings,
  3. Any additional kernel args or runtime setup needed to make ETM/TMC bind.

If possible, please share expected success logs and expected /sys/bus/coresight/devices node list for Orin Nano.

Thanks a lot.

7

Hi,
We have tested it on AGX Orin developer kit:
Are there any processor tracing support on Jetson AGX Orin - #9 by WayneWWW

Will check with the teams and see if it works the same on Orin Nano.

8

Thanks for your help! We are looking forward to your team’s feedback on whether the Orin Nano supports the same tracing features. Hope to hear from you soon.

9

## Jetson Orin Nano: ETM trace not producing payload (only 64-byte AUXTRACE)

I’m trying to collect ETM control-flow trace with `perf` on Jetson Orin Nano, but I cannot get valid ETM payload.
Below is the current status and reproduction details.

1) Device/PMU enumeration status

CoreSight devices currently visible under `/sys/bus/coresight/devices`:

  • `etm0 … etm5`
  • `funnel0 … funnel4`
  • `replicator0`
  • `tmc_etf0`

Perf side:

  • `perf list | grep -i cs_etm` shows `cs_etm//`
  • `/sys/bus/event_source/devices/cs_etm` exists

2) Key boot/runtime errors

Boot log contains:

  • `coresight-tmc: probe of 24070000.etr failed with error -13`

CBB errors also appear:

  • `Error:cbb-fabric, Errmon:2`
  • `Error Code: SLAVE_ERR`
  • `MASTER_ID: CCPLEX`
  • `Protection: Unprivileged, Non-Secure, Data Access`
  • Example fault addresses: `0x27840fe0 / 0x27940fe0 / 0x27a40fe0 / 0x27b40fe0`

3) ETM register status (example)

  • `trcauthstatus = 0x88`
  • `trcoslsr = 0xa`
  • `trcpdsr = 0x23` (on some cores)

I’m not sure whether `0x88` means non-secure ETM debug/trace is disabled on this platform.

4) Reproduction setup

Since ETR is unavailable, I disabled ETR in DTB and tried to sink ETM to ETF.

Commands:

```bash
sudo perf record -vv -o perf.data -e cs_etm// -C 0 – timeout 3s taskset -c 0 ./branch_demo
sudo perf report -D -i perf.data | rg “CoreSight ETM Trace data|AUXTRACE|SAMPLE events” -n
```

5) Observed result

Results are consistently:

  • `PERF_RECORD_AUXTRACE_INFO` exists
  • `PERF_RECORD_AUXTRACE` exists
  • but `CoreSight ETM Trace data: size 64 bytes`
  • `SAMPLE events: 0`
  • `perf script --itrace=…` has no output

So there is AUXTRACE metadata/header, but almost no valid ETM payload.

6) Question

Does `trcauthstatus = 0x88` on Orin Nano mean non-secure ETM trace is unavailable by default?
If yes, what is the official way to enable ETM trace so that real ETM payload can be collected?

11

Hi,
We have checked and confirmed CoreSight is not enabled on Jetpack 6 Orin. It is enabled only on Jetpack 5 Orin with the device tree:

/hardware/nvidia/soc/t23x/kernel-dts/tegra234-soc/tegra234-soc-coresight.dtsi
/hardware/nvidia/soc/t23x/kernel-dts/tegra234-soc/tegra234-soc-base.dtsi

Please note it.

13

Hi @1209762583 .

Please refer below steps:

Host Side (Ubuntu)
  1. Sync the L4T source tree for the target Jetson release:
cd Linux_for_Tegra/source
./source_sync.sh -t jetson_36.4.3
  1. Apply the CoreSight kernel patch to enable trace support:
cd Linux_for_Tegra/source/kernel/kernel-jammy-src
git apply coresight.patch
  1. Apply the device-tree overlay patch for CoreSight nodes:
cd Linux_for_Tegra/source/hardware/nvidia/t23x/nv-public
git apply overlay.patch

  1. Drop in the CoreSight SoC device-tree include:

    • Add tegra234-soc-coresight.dtsi into Linux_for_Tegra/source/hardware/nvidia/t23x/nv-public/nv-soc/.

  2. Build the Jetson Linux Kernel, NVIDIA Out-of-Tree Modules, and DTBs by following the Kernel Customization guide.

  3. Enable non-secure debug access in BCT (slot B) by editing Linux_for_Tegra/bootloader/generic/BCT/tegra234-br-bct_b-p3767-0000-l4t.dts:

-        SecureDebugControlNoneEcid = <0x0>;
+        SecureDebugControlNoneEcid = <0x10>;
  1. Apply the same BCT change to the primary slot in Linux_for_Tegra/bootloader/generic/BCT/tegra234-br-bct-p3767-0000-l4t.dts:
-        SecureDebugControlNoneEcid = <0x0>;
+        SecureDebugControlNoneEcid = <0x10>;
  1. Flash the device
Target Device (Jetson Orin Nano)
  1. Ensure the running kernel is built with CoreSight enabled. Verify options (example from a working build):
$ zgrep CORESIGHT /proc/config.gz

CONFIG_CORESIGHT=y
CONFIG_CORESIGHT_LINKS_AND_SINKS=y
CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
# CONFIG_CORESIGHT_CATU is not set
# CONFIG_CORESIGHT_SINK_TPIU is not set
# CONFIG_CORESIGHT_SINK_ETBV10 is not set
CONFIG_CORESIGHT_SOURCE_ETM4X=y
CONFIG_CORESIGHT_STM=y
# CONFIG_CORESIGHT_CPU_DEBUG is not set
# CONFIG_CORESIGHT_CTI is not set
# CONFIG_CORESIGHT_TRBE is not set
  1. Confirm CoreSight devices are exposed under sysfs:
$ ls /sys/bus/coresight/devices

etm0  etm1  etm2  etm3  etm4  etm5  funnel0  funnel1  funnel2  funnel3  funnel4  stm0  tmc_etf0
  1. Follow the forum topic to build perf with the CoreSight Embedded Trace Extension.
  2. Run a minimal perf trace test (adjust CPU and sink name to match your system):
sudo perf record -m 32768 -o perf.data -e cs_etm/@tmc_etf0/ -C 0 -- taskset -c 0 sleep 1
sudo perf report -D -i perf.data

Thanks

coresight.patch (1.2 KB)
overlay.patch (1.1 KB)
tegra234-soc-coresight.dtsi (11.2 KB)

14

@DavidDDD I have a similar issue. Right now I’m able to have all the ETMs detected and working, but struggling with the ETR sink. My goal is to have end-to-end trace working with decent sized trace buffer, thus I really need ETR. I’m using a Orin AGX, with L4T 35.6.4, where there’s coresight.dtsi shipped. I’m able to patch dts and kernel config to enable all the coresight components and drivers, however, I’m seeing the same warning in kernel log:

[   16.680327] coresight-tmc: probe of 24070000.etr failed with error -13

I’m using L4T 35.6.4 and Kernel 5.10.216. From the kernel source, it appears that the ETR registers were not setup properly:

static inline bool tmc_etr_has_non_secure_access(struct tmc_drvdata *drvdata)
{
	u32 auth = readl_relaxed(drvdata->base + TMC_AUTHSTATUS);

	return (auth & TMC_AUTH_NSID_MASK) == 0x3;
}

It appears that the “Non-secure invasive debug” fields of TMC_AUTHSTATUS is not set to enabled. ref:

Per your previous reply, I’ve already patched both bootloader/t186ref/BCT/tegra234-br-bct-p3701-0000.dts and bootloader/tegra234-br-bct_b-p3701-0000.dts, given I’m using Orin AGX:

It seems that they are not enough to enable “Non-secure invasive debug” on the ETR component. Please advise. It’s very frustrating that there are many Orin coresight related threads on the forum, but none of them worked for me.

E.g., I also tried to follow instructions from @Bibek 【Jetson AGX Orin】How to config to trace to system RAM using Coresight with ETR in Arm Development Studio by Arm DStream debugger? - #6 by movelab , but the section around modifying tegra234-mb1-bct-misc-p3701-0000.dts is definitely wrong. There’s no coresight.cfg_etr_mc_wr/rd_ctrl fields in the t23x BCT, as it’s already moved away from the t19x’s plaintext format to the new device-tree style. I’ve tried to set /misc/coresight/coresight.cfg_etr_mc_wr/rd_ctrl in dts, but tegrabct_v2 complains about unknown fields in MB1 BCT.

Nvidia, it’s pretty clear that the SW side of the Jetson Orin needs to be improved. I’ve spend days if not weeks trying to get coresight working, to no avail.

15

Hi @ganboing
This thread is for Orin Nano. Please create a new thread for AGX Orin. And please follow above steps to try Jetpack 6.2.2 r36.5.