Tx2 PCI-E DeviceDriver -- ioremap() succeeds but not all BAR space is accessible (kernel paging error)

My driver fails when I access the “upper” part of my virtualzed PCI-E space (Bar0 - 2MB MemWindow)
Driver is fine on x64 Ubuntu…something ARM specific?

Crash: handle kernel paging request at virtual address ffffff801020400c
Kernel: 4.9.253-tegra #1 SMP PREEMPT Mon Jul 26 12:19:28 PDT 2021 aarch64 aarch64 aarch64 GNU/Linux

Pertinent info below…appreciate any help…

Mike

map results/virtual addresses

0: Mapped global memory bar to ioremap retured ffffff8010200000 with size 2048 KB
_mil aim_mil0: Mapped IO memory bar to ffffff800b230000 with size 64 KB
[ 4.613946] aim_mil aim_mil0: Allocated shared memory to ffffff8014001000 with size 16384 KB

Crash Dump

4.654654] Unable to handle kernel paging request at virtual address ffffff801020400c
[ 4.664498] Mem abort info:
[ 4.675722] CPU2: shutdown
[ 4.676375] psci: CPU2 killed (polled 4 ms)
[ 4.744559] ESR = 0x96000061
[ 4.751733] Exception class = DABT (current EL), IL = 32 bits
[ 4.760126] SET = 0, FnV = 0
[ 4.764178] EA = 0, S1PTW = 0
[ 4.774531] Data abort info:
[ 4.777678] ISV = 0, ISS = 0x00000061
[ 4.781981] CM = 0, WnR = 1
[ 4.785428] swapper pgtable: 4k pages, 39-bit VAs, pgd = ffffff800a200000
[ 4.793023] [ffffff801020400c] *pgd=00000002371fe003, *pud=00000002371fe003, *pmd=0000000215e87003, *pte=00e8000040204707
[ 4.812628] Internal error: Oops: 96000061 [#1] PREEMPT SMP
[ 4.818221] Modules linked in: aim_mil(O+) overlay nvgpu ip_tables x_tables
[ 4.825296] CPU: 5 PID: 3154 Comm: systemd-udevd Tainted: G O 4.9.253-tegra #1
[ 4.833670] Hardware name: storm (DT)
[ 4.837345] task: ffffffc1a3086200 task.stack: ffffffc1a2274000
[ 4.843340] PC is at api_io_bm_rt_act_ini+0x7c/0x300 [aim_mil]
[ 4.849241] LR is at api_io_bm_rt_act_ini+0x208/0x300 [aim_mil]
[ 4.855179] pc : [] lr : [] pstate: 00400045
[ 4.862593] sp : ffffffc1a22777c0
[ 4.865915] x29: ffffffc1a22777c0 x28: ffffff800117ac98
[ 4.871026] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 4.876985] x27: ffffffc191300000 x26: ffffffc191310000
[ 4.882333] x25: ffffffc191312000 x24: 0000000020000000
[ 4.887680] x23: ffffffc191000000 x22: 0000000000000014
[ 4.893028] x21: 000000000000000c x20: 0000000000000000
[ 4.898375] x19: 0000000000000001 x18: 0000000000000001
[ 4.903722] x17: 0000000000000001 x16: 0000000000000007
[ 4.909069] x15: ffffffffffffffff x14: ffffff800a16edd8
[ 4.914417] x13: ffffff800a16ea17 x12: 0000000000000000
[ 4.919785] x11: 0000000000000006 x10: 00000000000003c2
[ 4.925135] x9 : 0000000000000000 x8 : ffffffc1b7052977
[ 4.930484] x7 : 0000000000000000 x6 : 0000000000000001
[ 4.935832] x5 : 0000000000000000 x4 : ffffffc1b5fecbe8
[ 4.941179] x3 : ffffffc1b5fecbe8 x2 : ffffff8010200000
[ 4.944449] bcpu-throttle-alert cooling device registered.
[ 4.945404] mcpu-throttle-alert cooling device registered.
[ 4.948151] gpu-throttle-alert cooling device registered.
[ 4.949267] gpio tegra-gpio wake18 for gpio=101(M:5)
[ 4.953080] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 4.973578] x1 : 000000000000400c x0 : ffffff801020400c

[ 4.980426] Process systemd-udevd (pid: 3154, stack limit = 0xffffffc1a2274000)
[ 4.987615] zram: Added device: zram0
[ 4.991420] Call trace:
[ 4.992698] zram: Added device: zram1
[ 4.997595] [] api_io_bm_rt_act_ini+0x7c/0x300 [aim_mil]
[ 5.004540] [] api_io_bm_ini+0xc4/0x190 [aim_mil]
[ 5.012515] [] api_sys_reset+0x738/0x868 [aim_mil]
[ 5.020599] [] api_main_init_and_reset+0x200/0x2b8 [aim_mil]
[ 5.029642] [] mil_hw_initialize_tsw+0x6c/0xa0 [aim_mil]
[ 5.038386] [] aim_pci_mil_init_tsw+0xb0/0x130 [aim_mil]
[ 5.047198] [] aim_pci_device_init_mil+0x80/0x90 [aim_mil]
[ 5.056002] [] aim_pci_probe+0x1f8/0x400 [aim_mil]
[ 5.064077] [] pci_device_probe+0xa4/0x140
[ 5.071426] [] driver_probe_device+0x298/0x448

lspci info

01:00.0 Memory controller: MyDevice Device 4100
Subsystem: MyDevice Device 0011
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 382
Region 0: Memory at 40200000 (64-bit, non-prefetchable) [size=2M]
Region 2: Memory at 40100000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [60] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [100 v1] Device Serial Number 00-00-00-01-01-00-0a-35
Kernel driver in use: my_driver
Kernel modules: my_driver

Sorry for the late response, is this still an issue to support?

Thanks

I think I got it working. It had to do with alignment issues.

Does the aarch64 have an alignment requirement when accessing physical PCI memory via ioremap()?

make sure that it is DW (i.e. 4 bytes) aligned and that should work for you.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.