Hello,
We are using the TX1 (running L4T) in our carrier board. At this point we are finding interference between the TX1 and other systems when connecting a PCIe device to the TX1. Are there any options that allow us to change the PCIe clocks or easily force a device into a low power mode?
Thanks,
Robb
PCIe supplies REFCLK to end point and its a fixed 100 MHz clock. bit rate on Tx/Rx lanes depend on the speed (Gen-1/Gen-2) at which link is operating. So, I don’t think these two at least can’t be manually controlled.
As far as low power states are concerned, what are the different device and link low power states supported by your end point? and accordingly it can be put to those states.
Also, can you give more info on ‘interference between TX1 and other systems’ part?
Also, which release are you using (24.1/24.2 etc…)? accordingly we can check if Spread Spectrum is enabled or not.
Thanks for the quick reply.
I will have to look into which, if any, low power states supported by our device. We are using wireless network adapters and find they are causing EMI in the GPS frequency range. We’ve managed to rule out interference directly from the device, so we’re investigating the PCIe bus. I am hoping there is a method to tune the frequency or power to observe changes in interference.
We are using the R24.2 release.
I’m a colleague of Robb, approaching this problem from an electrical engineering (board design) angle. In browsing the TRM version 1.1 (the most recent), in addition to finding all the standard PCIe control registers for reduced voltage swing, I noticed a suite of private NVIDIA registers which appear to control the low-level physical layer functions. They’re listed in the TRM beginning from section 34.4.8.30 through 34.4.8.44. However, no description of how to set the bit fields is given. Can NVIDIA please explain the function of these registers and how to set them?
For another description of the issue we are seeing, please see my post here: https://devtalk.nvidia.com/default/topic/1000355/controlled-impedance-for-pcie-on-carrier-board/
I can also report that we see similar GPS desense impact when using the Nvidia-designed Jetson carrier board, not just our own custom carrier board.
Putting the endpoint into other low-power states seems like a good debug step, too. I’ll add that do our todo list. It’s an Atheros AR9590 based design.
I can also report that we’ve tried disabling the TX and RX lanes on PCIe while keeping REFCLK running, and the desense disappears. So this led us to look more at the amplitude and other factors we could try adjusting on the TX/RX lanes. While that may not be the end solution, it’ll help us confirm where the desense is actually coming from.
Thanks,
Mike
One thing I’ve wondered about is if the -6dB deemphasis/preemphasis is really correct (and this could change EMI). In PCIe gen. 1 -3.5dB was fixed. In PCIe gen.2 -6dB was added, but this was only intended for PCIe slots intended to have long copper runs (it was always intended to remain at -3.5dB for normal copper runs…as motherboards became larger and runs became longer -6dB was introduced for that purpose when gen. 2 was born). Should -3.5dB deemphasis work this could drop EMI. I don’t know how to try -3.5dB, but it seems copper runs are short enough -3.5dB should work.
robb_n / MikeB226,
Can you please provide steps to reproduce the issue with Nvidia designed Jetson carrier board?
I’ll get back to you about registers in TRM.
Hi vidyas,
The majority of our tests were done with our own PCB instead of the Jetson carrier board. That said, we still encountered interference with the GPS module from the TX1 on the Jetson. I’m trying to follow up on the exact steps we used.
The easiest test to run would be to obtain a GPS module and monitor the carrier-to-noise ratio (CNO). Our PCIe device is an Atheros card (PCIe Gen 1).
I have also attempted to update the PCIe drivers (in the R24.2 L4T release) to vary the amplitude and timing of the PCIe REFCLK, TX and RX lanes. Unfortunately, I have not managed to observe any changes when scoping the lines. Maybe you can provide some guidance and feedback. Here is what I’ve attempted so far:
This was accomplished by updating the CLK_RST_CONTROLLER_PLLE_SS_CNTL_0 register to set the SSCBYP, INTERP_RESET and BYPASS_SS bits. I made sure that USE_PLLE_SS was defined to 0 in both include/linux/platform/tegra/clock.h and drivers/platform/tegra/tegra12_clocks.c.
I wasn’t able to measure any change in REFCLK after this change.
It looks like this could be accomplished with the T_PCIE2_RP_LINK_CONTROL_STATUS_2 register by setting the TRANSMIT_MARGIN bit. However, updating this to a value of 011b from the default did not have any noticeable changes. I would expect a significant change in the amplitude of the signal.
I tested this change both with and without modifying the CTL_TX_MARGIN_OVERRIDE bit in the T_PCIE2_RP_PRIV_XP_LCTRL_2 register.
If these aren’t the right registers, I am hoping the registers in T_PCIE2_RP_ECTL_1_R1 (section 34.4.8.30 in the TRM) will allow us to tweak the timing parameters for TX.
This was accomplished by setting the DEEMPHASIS_STRAP and ENFORCE_DEEMPHASIS bits in the T_PCIE2_RP_PRIV_XP_LCTRL_2 register.
Again, we did not notice any change in the signals. The status from the CURRENT_DEEMPHASIS_LEVEL bit in the T_PCIE2_RP_LINK_CONTROL_STATUS_2 did not seem to indicate the device was configured for 3.5dB deemphasis.
Hi vidyas,
Is there an update on the TRM registers?
I’ve followed up with the test steps used with the Jetson carrier board and the previous steps listed were correct. Several WLAN PCIe cards were used and we found similar results, so I’m sure it’s reproduceable with another card if an Atheros isn’t available. A uBlox GPS module was used to measure any interference by reading the CNO.
Thanks,
Robb
>> Is there an update on the TRM registers?
Apologies for the delay in reply. I’m yet to hear back from corresponding folks.
BTW, since the device is operating at Gen-1 speed, I don’t think de-emphasis settings are relevant here.
Also, we are interested to know how did you manage to rule out interference from device? (comment #3)
Hi Robb_n / MikeB226,
Have you tried other type of the PCIE devices(not WLAN card)?
I mean do you rule out the desense from the 2.4G wifi source?
Some actions to try:
Hi Vidyas/Tyzhang,
Also, we are interested to know how did you manage to rule out interference from device? (comment #3)
To isolate if it’s our device, we tried a few devices that show similar results when measuring the GPS signal.
- Change non PCIE-WLAN device
So far all of our tests were with different WLAN devices. We have ordered some non PCIE-WLAN units to test. I will update you on the results when they become available.
- Disable WIFI radio if you can’t find other devices
We’ve started tests for this. I will provide the results when they are available.
- Add metal foil(for shielding) on the WLAN card(focus on antenna/socket area) if you can’t disable its radio
We have tested shielding the PCIE device. The resulting desense is similar to the unshielded device.
Hi robb_n,
What’s the length of your PCIe board? Do you have other much shorter or longer PCIe devices to check if the interference is less with them plug in?
Artificially make traces longer with a riser card and see what happens to interference amplitude.
A few updates:
- Change non PCIE-WLAN device
We’ve used a PCIe network (ethernet) adapter instead of a WLAN card and observed similar desense results.
- Disable WIFI radio if you can’t find other devices
No change was observed with this change. The change involved simply bringing down that particular interface in linux and rmmod-ing the appropriate drivers. The PCIe device still enumerated with the lspci command.
I suspect there is no change compared to previous tests because there was never an active WiFi connection on that interface.
- Use spectrum probe to find out the exact peak frequency points, which locates in GPS range
This is still on our list of items to measure.
Artificially make traces longer with a riser card and see what happens to interference amplitude.
We’ve obtained similar desense with PCIe enabled on our custom carrier board and the Jetson devkit. These have different trace layouts. I understand that there may be other factors affecting interference since the two systems have different hardware, but our measurements don’t show anything very different.
We have other tests underway that involve changing the hardware configuration, assembly and parts to measure the effect on GPS. However, the software/driver portion is a bit blocked.
Ideally, I would like to change the PCIE data TX amplitude and timing parameters. I have made some changes that are described in comment 7. I expected to observe changes in the TX/RX eye-diagrams after these changes, but it looks the same.
Just to verify, does the above mean that a riser card had no significant effect? Obviously one of the checks is if the GPS is failing, but was a spectrum analyzer used for this particular test to show amplitude did not change significantly in the GPS’s band? If so, then I’d start to think that although the issue is related to PCIe being enabled, that it isn’t the actual PCIe data/control lanes producing the the interference (perhaps there are related functions when PCIe is enabled, but not directly part of the data lanes or control…e.g., an indirectly related clock signal…in which case traces to the clock would be the culprit and may not connect directly to a wire distributed to the PCIe slot).
Hi robb,
Seems the interference is caused by PCIe lanes because of the same result with non-WIFI board.
Besides the registers tuning, there is some possible HW resolution. The length of your PCIe board is about 5cm, which is about 1/4 of wave length of GPS (19cm), so the ground planes (of PCIe board/carrier board) + BTB connector, that could form a good microstrip patch antenna of GPS frequency. To eliminate this interfere you can use a fully metal grounded cover to PCIe board or manually extend the board length by a metal foil connected to ground plane of board, which could change the character of this microstrip patch antenna.
Hi Robb,
Do you have any progress?
Hi Trumany,
We are still having problems with PCIe and GPS.
Since the last email, we have tried a few other tests:
Other PCIe cards by other brands show similar levels of GPS desense.
We are also able to reproduce similar results with the TX2 module.
Update the mselect and emc clock rates to different values.
I noticed the clock rates are different before and after the PCIe card enumerates. There was no improvement to GPS desense after making these changes.
Disabling those lanes was achieved by updating the AFI_PME_0 register to set the SM2TMS0C1_PME_TO and SM2TMS0C0_PME_TO bits. We verified the the REFCLK was still running.
After this change there was no interference with GPS. This reinforces our prediction that the problem is strongly related to PCIe. However, this is not a solution, since the PCIe bus is unusable in this state.
Other items we have in the pipeline include:
All of the previous tests were conducted with Gen 1 cards since that is the only available speed for the WLAN card we wish to use. Still, it would be interesting to know if this problem goes away at a higher clock rates.
Have you had a chance to find out if additional documentation is available for the private registers in the TRM? I am specifically looking to change the amplitude and timing parameters of the PCIe TX and RX signals.
Different PCIe cards have roughly the same level of desense, but it is not always consistent. The eye-diagrams of TX and RX are also different. Since there is some correlation, we would like to tune this and observe changes in the GPS signal.
You’d better to find out the spot of desense source directly. You can use the close field probe (such as N9311X-100) to locate the EMI area. Then apply the shielding protect or move the GPS to a better location.
Do not suggest to tune the PCIe driving as the values are already qualified by our characterization team and the tuning could cause potential risks.