Hey everyone,
I’m trying to achieve something with the xavier NX and failed over and over, miserably. In the question, I’ll give a description of what I’m trying to achieve, and step-by-step, how I failed.
My goal
I have a PCIe device I need to access to NX’s memory(This device does not have a specific driver, I’ll need to develop one for it). This device I’m talking about, and the NX, the two will be a compact device. Since, our configuration is solid and will not change, we decided to build our communication on BAR matching PCIe packets.
To achieve this, in the NX, we need to reserve a specific memory address and direct PCIe packets to that memory with inbound address translation unit of the pcie controller, then redirect pcie memory to that reserved space for this communication to be inbound. (I know this is achievable, we previously did the same with some other chip.)
Step-1: Reserving Physical Memory
So I added:
my_reserved: my_pciemem@ac200000 {
reg = <0x0 0xac200000 0x0 0x08000000>;
};
to the device tree (tegra194-soc-base.dtsi). For choosing this specific address, I got help on the device tree node “memory” by:
xxd /proc/device-tree/memory/reg
Question about step 1:
When I checked /proc/iomem, there is no memory in the name of my choice and the memory I try to get is still the part of System RAM. Is there a way to check if this operation is successful?
Step-2: Programming the pcie iatu
Assuming the first step is successful, I moved on with programming iATU. After all calls, I checked the return value of functions for any errors, and got none. After programming the pcie controller, I checked the registers programmed and convinced myself that iATU programming is succesful.
However, when I let the other device start writing on my pcie memory, I got lots of SMMU errors. Here is just one, the others are the same except iova:
t19x-arm-smmu 12000000.iommu: Unhandled context fault: smmu0, iova=0x1f40000000, fsynr=0x20011, cb=1, sid=91(0x5b - PCIE5), pgd=0, pud=0, pmd=0, pte=0
Step-3: Disabling SMMU
When I searched the forum with the smmu error message, most popular answer I got is disabling SMMU. So I did that, at least I hoped I did. For disabling it, I did exactly the same with:
However, the errors still persist. However, I believe fsynr= value is changed.
Question about step 3:
Can I check if I disabled SMMU correctly for PCIE5?
So my question is: What have I done wrong or missing to not achieve my goal? Any corrections on any step is appriciated.