kernel-4.4/drivers/platform/tegra/mc/isomgr.c: isomgr_init() fails to initialize

Hi,

I currently face the problem that the isomgr_init() fails to initialize on our custom carrier board

....
[    1.516220] bwmgr: round_rate = 0, bwmgr.emc_min_rate = 0
[    1.522044] bwmgr: round_rate = 0, bwmgr.emc_max_rate = 0
[    1.527749] bwmgr: bwmgr.status = true
....
[    1.768500] isomgr_init(): iso emc max clk=0KHz
[    1.768502] isomgr_init(): max_iso_bw=0KB

Results in audio, ethernet, sound fails to initialize too

...
[    1.872247] ---[ end trace cbbc6543b6c23fe4 ]---
[    1.872252] Call trace:
[    1.872258] [<ffffffc0007c4470>] __tegra_isomgr_register+0x30c/0x4d4
[    1.872301] [<ffffffc0007c4654>] tegra_isomgr_register+0x1c/0x2c
[    1.872323] [<ffffffc000417748>] tegra_camera_probe+0x270/0x3d8
[    1.872341] [<ffffffc000532948>] platform_drv_probe+0x50/0xbc
[    1.872347] [<ffffffc000530468>] driver_probe_device+0xc8/0x408
[    1.872351] [<ffffffc000530844>] __driver_attach+0x9c/0xa0
[    1.872359] [<ffffffc00052e4e4>] bus_for_each_dev+0x58/0x98
[    1.872364] [<ffffffc00052ff10>] driver_attach+0x20/0x28
[    1.872369] [<ffffffc00052fa80>] bus_add_driver+0x1f0/0x294
[    1.872374] [<ffffffc00053162c>] driver_register+0x68/0x108
[    1.872378] [<ffffffc000532850>] __platform_driver_register+0x54/0x5c
[    1.872385] [<ffffffc000fe59d8>] tegra_camera_init+0x18/0x20
[    1.872400] [<ffffffc000081ab4>] do_one_initcall+0xc8/0x1c0
[    1.872423] [<ffffffc000fbab60>] kernel_init_freeable+0x1d0/0x274
[    1.872444] [<ffffffc000a5cd6c>] kernel_init+0x10/0xe0
[    1.872457] [<ffffffc000084e10>] ret_from_fork+0x10/0x40
...

It looks like there is no clock privided for EMC module but TX2 Module DataSheet says “The Jetson TX2 requires no external clocks for operation, all system clocks are generated within the module”

Everything works fine on developent kit.

...
[    1.412091] bwmgr: round_rate = 40800000, bwmgr.emc_min_rate = 40800000
[    1.419084] bwmgr: round_rate = 1866000000, bwmgr.emc_max_rate = 1866000000
[    1.426342] bwmgr: bwmgr.status = true
...
[    1.685654] isomgr_init(): iso emc max clk=1866000KHz
[    1.685657] isomgr_init(): max_iso_bw=26870400KB

I tested with stock R27.1 release

The bwmgr node in DTB is the same on my custom carrier board and Dev Kit.

bwmgr {
		compatible = "nvidia,bwmgr";
		clock-names = "emc";
		clocks = <0xd 0x3a>;
		status = "okay";
	};

Does anyone experience the same issue?
Do I miss something and do you have any suggestion to make it work?
Could we find the docs for Terga clock, isomgr sybsystem?

Thanks,

Hi muilevan
Can it boot to OS?
Could you dump below value.

sudo cat /sys/kernel/debug/tegra_bwmgr/emc_max_rate
sudo cat /sys/kernel/debug/tegra_bwmgr/emc_min_rate
sudo cat /sys/kernel/debug/tegra_bwmgr/emc_rate

Hi ShaneCCC,

The system can boot to OS prompt with a lot of Calltrace as above and the isomgr client (eg: audio, camera, ethernet,…) does not work.

Here is the info you requested

On our carrier board:

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_max_rate
[sudo] password for nvidia:
0
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_min_rate
0
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_rate
0

On Dev Kit

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_max_rate
[sudo] password for nvidia:
1866000000
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_min_rate
40800000
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/tegra_bwmgr/emc_rate
1866000000

This is the same as debug log I added above.

Thanks,

Hi muilevan
Please help to dump below information again. If they are zero too you lost the DVFS table for you board.

cat /sys/kernel/debug/bpmp/debug/clk/emc/rate
cat /sys/kernel/debug/bpmp/debug/emc/possible_rates.

Hi ShaneCCC,

This is info of TX2 on our carrier board

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/bpmp/debug/clk/emc/rate
0
nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/bpmp/debug/emc/possible_rates
cat: /sys/kernel/debug/bpmp/debug/emc/possible_rates: No such file or directory

Could you let me know where DVFS is stored? (eg. in EEPROM on Dev Kit or TX2 module?)
The same TX2 module works on Dev Kit but not on our carrier board.

Sorry for typo, should be below path.

/sys/kernel/debug/bpmp/debug/emc

Could you confirm use the same CPU module on different carrier board get different result?

Hi ShaneCCC,

Yes, actually we have only 1 TX2 module for evaluation now, the behavior is different between Dev Kit and our carrier board

On our carrier board, there is no such emc directory under /sys/kernel/debug/bpmp/debug/

root@tegra-ubuntu:~# find /sys/kernel/debug/bpmp/debug/ | grep -w emc
/sys/kernel/debug/bpmp/debug/clk/emc
/sys/kernel/debug/bpmp/debug/clk/emc/dvfs
/sys/kernel/debug/bpmp/debug/clk/emc/pto_counter
/sys/kernel/debug/bpmp/debug/clk/emc/all_children
/sys/kernel/debug/bpmp/debug/clk/emc/children
/sys/kernel/debug/bpmp/debug/clk/emc/possible_parents
/sys/kernel/debug/bpmp/debug/clk/emc/parent
/sys/kernel/debug/bpmp/debug/clk/emc/max_rate
/sys/kernel/debug/bpmp/debug/clk/emc/min_rate
/sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
/sys/kernel/debug/bpmp/debug/clk/emc/state
/sys/kernel/debug/bpmp/debug/clk/emc/rate

On Dev Kit, everything is there.

root@tegra-ubuntu:/sys/kernel/debug/bpmp/debug/emc# ls -1
cc_debug
cc_seq
dll_prelock_version
dram_temp_range
dram_therm
over_temp_state
periodic_calibration
pll_lib
possible_rates
stats
tables
tables_info
timer_period_training
root@tegra-ubuntu:/sys/kernel/debug/bpmp/debug/emc# cat possible_rates 
40800 68000 102000 204000 408000 665600 800000 1062400 1331200 1600000 1866000 (kHz)
root@tegra-ubuntu:/sys/kernel/debug/bpmp/debug/emc# cat /sys/kernel/debug/bpmp/debug/clk/emc/rate
665600000

Thanks,

Hi,

Please share the complete bootlog ( not only kernel) and flashing log.
Also make sure that you have applied the change mentioned in the below link for custom board flashing

regards
Bibek

Hi Bibek,

Please find the attached for flashing/booting logs

I just concern about whether MB1/MB2/ATF/Cboot are generic enough or rely on some components on Dev Kit (Eg. I2C devices, GPIOs,…) that are reduced on our custom carrier board.

Thanks,

tx2_on_dev_kit_flashing_and_boot_log.txt (93 KB)
host_flashing_log.txt (192 KB)
tx2_on_custom_carrier_boot_log.txt (170 KB)

Hi Muilevan,

Only MB1 cfg files and kernel dt files are platform dependent.
Can you try making the following change and provide new logs.

  1. since your platform does not have the expander i2c slave @ 0x74. can you remove all instances of it from kernel dt. Just add the below lines to kernel final dts and rebuild the dtb. Replace the new dtb to Linux_for_Tegra/kernel/dtb/ folder
    File: tegra186-quill-p3310-1000-c03-00-base.dtb

i2c@3160000 {
/delete-node/ gpio@74;
/delete-node/ gpio@77;
};

&vdd_hdmi {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};

&en_vdd_ts_hv_3v3 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};

&en_vdd_ts_1v8 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};

&dis_vdd_1v2 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_avdd_disp_3v3 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_mdm_pwr_3v7 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_vdd_cam {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_vdd_cam_1v2 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_vdd_cam_hv_2v8 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_vdd_disp_1v8 {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&en_vdd_sys {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&vdd_fan {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};
&vdd_usb2_5v {
compatible = “regulator-fixed”;
/delete-property/ gpio;
/delete-property/ enable-active-high;
regulator-always-on;
};

  1. I want to see BPMP logs for emc issue. Make below change to bpmp dtb and reflash
    File: tegra186-a02-bpmp-quill-p3310-1000-c01-00-te770d-ucm2.dtb from Linux_for_Tegra/bootloader/ folder
    convert to dts :
    dtc -I dtb -O dts tegra186-a02-bpmp-quill-p3310-1000-c01-00-te770d-ucm2.dtb > tegra186-a02-bpmp-quill-p3310-1000-c01-00-te770d-ucm2.dts

Make the change here:
serial {
- port = <0x00000007>;
+ port = <0x000000ff>;
has_input;
};

  1. And then flash. After kernel boots.
    cat /dev/ttyBPMP0 > bpmp.log
    share boot log and bpmp.log

Hi Bibek,

Thanks for your response.

Please file the attached for the logs with the changes you suggested.

Looks like this is problem:

...
EMC: Invalid strap
tegra_emc_dt_parse_pdata: Couldn't find EMC device tree node
t186_emc_init: Missing emc dt table!
...

Could you suggest how to fix this?

Thanks,

tx2_on_dev_kit_with_bibek_changes_log.txt (79.6 KB)
tx2_on_custom_carrier_with_bibek_changes_log.txt (167 KB)

Hi Muilevan,

Now my next question is:
What are you doing with these two pins coming out of Jetson TX2 connector, in your custom base board?
Pin Name Ball number
UART0_RTS G11
UART1_TX D9

Value on these two pins decide the pin strap. Its recommended to not use these two pins.

regards
Bibek

Hi Bibek,

Thank you for your information.
The EMC module is now working on our carrier board after isolating UART0_RTS and UART1_TX. These 2 pins are connected to a multiplexer on our carrier board that causes unexpected value for EMC pin strap detection.
We assume these pins are used for serial console only.
In my opinion, the UART pins should not be used as pin strap (dedicated GPIOs instead, I think).

By the way, Could you please share where we can find the official documentation for TX2 pin strap?

Thanks,

Good to know that its working.
You can get strap info here http://developer.nvidia.com/embedded/dlc/jetson-tx2-oem-product-design-guide

thanks
Bibek

Hi Bibek,

Thanks for your information.

The documentation is very helpful for us.

Thanks,

Hello,

Could someone explain in a little more detail how to resolve this issue?

I also have this issue where emc_min/max/rate read as zero on a custom carrier, and memory performance is terrible (apparently the zero value means “really slow”). I am using an auvidea J100 board, which breaks out UART0 and UART1 to connectors (including G11 and D9).

What do I need to do with those pins to bring back the EMC clock settings?

Neither pull up or down those two pins.

The J100 carrier has level shifters to bring out UART0, UART1 and UART2 at 3.3V. The level shifters must be pulling these pins and causing the EMC misconfiguration.

I could sever the traces to fix this, but that would sacrifice precious I/O, we rely on the UARTs for sensor inputs.

Is there any way to reconfigure the TX2 to use different pins for the strapping procedure?

Can you try not pulling up the pins early during boot, then the problem will not occur.
for example, powering on the level shifter only in kernel?

By the way, TX2 has 7 UART controllers to choose from.

This is an off-the-shelf carrier, the level shifters seem to be powered with everything else.

TX2 brings out 5 UARTs, TX1 only 3. So a board designed to use TX1 UARTs can’t just take a TX2 as a drop-in replacement without a hardware revision. Just some dashed hopes of being able to upgrade quickly.