Request information about pci_regs.h defines

Hi,

I met these errors when I compiled (gcc-4.8 with arm64) the Linux RIFFA driver on the JTX1 :

NAME: 'riffa'
   MAJNUM: '100'
VENDOR_ID0: '10EE'
VENDOR_ID1: '1172'
...
make[1]: Entering directory `/usr/src/linux-headers-3.10.67-g458d45c'
make[1]: Warning: File `/usr/src/linux-headers-3.10.67-g458d45c/arch/arm64/Makefile' has modification time 1.1e+08 s in the future
make[2]: Warning: File `scripts/Makefile.lib' has modification time 1.1e+08 s in the future
  CC [M]  /home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.o
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c: In function ‘fill_sg_buf’:
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:392:24: error: implicit declaration of function ‘kmalloc’ [-Werror=implicit-function-declaration]
  if ((sg_map = (struct sg_mapping *)kmalloc(sizeof(*sg_map), GFP_KERNEL)) == NULL) {
                        ^
...
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:403:4: error: implicit declaration of function ‘kfree’ [-Werror=implicit-function-declaration]
    kfree(sg_map);
    ^
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:419:3: error: implicit declaration of function ‘kcalloc’ [-Werror=implicit-function-declaration]
   if ((sgl = kcalloc(num_pages, sizeof(*sgl), GFP_KERNEL)) == NULL) {
   ^
...
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c: In function ‘allocate_chnls’:
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1000:25: error: implicit declaration of function ‘kzalloc’ [-Werror=implicit-function-declaration]
   sc->recv[i] = (struct chnl_dir *) kzalloc(sizeof(struct chnl_dir), GFP_KERNEL);
                         ^
...
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1200:76: error: ‘PCI_EXP_DEVCTL2_IDO_REQ_EN’ undeclared (first use in this function)
  error = pcie_capability_write_dword(dev,PCI_EXP_DEVCTL2,(devctl2_result | PCI_EXP_DEVCTL2_IDO_REQ_EN | PCI_EXP_DEVCTL2_IDO_CMP_EN));
                                                                            ^
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1200:76: note: each undeclared identifier is reported only once for each function it appears in
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1200:105: error: ‘PCI_EXP_DEVCTL2_IDO_CMP_EN’ undeclared (first use in this function)
  error = pcie_capability_write_dword(dev,PCI_EXP_DEVCTL2,(devctl2_result | PCI_EXP_DEVCTL2_IDO_REQ_EN | PCI_EXP_DEVCTL2_IDO_CMP_EN));
                                                                                                         ^
...
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c: In function ‘fpga_remove’:
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1445:61: error: ‘PCI_EXP_DEVCTL2_IDO_REQ_EN’ undeclared (first use in this function)
  pcie_capability_write_dword(dev,PCI_EXP_DEVCTL2,result & ~(PCI_EXP_DEVCTL2_IDO_REQ_EN | PCI_EXP_DEVCTL2_IDO_CMP_EN));
                                                             ^
/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.c:1445:90: error: ‘PCI_EXP_DEVCTL2_IDO_CMP_EN’ undeclared (first use in this function)
  pcie_capability_write_dword(dev,PCI_EXP_DEVCTL2,result & ~(PCI_EXP_DEVCTL2_IDO_REQ_EN | PCI_EXP_DEVCTL2_IDO_CMP_EN));
                                                                                          ^
cc1: some warnings being treated as errors
make[2]: *** [/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux/riffa_driver.o] Error 1
make[1]: *** [_module_/home/ubuntu/RIFFA_tmp/riffa_2.2.1/source/driver/linux] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-3.10.67-g458d45c'
make: *** [riffa.ko] Error 2

I succeeded to resolve it by adding the below code lines in riffa_driver.c because the missing defines are not present in pci_regs.h.

/* Temporary fix */
#include <linux/slab.h>
#define	PCI_EXP_DEVCTL2_IDO_REQ_EN 0x0100
#define	PCI_EXP_DEVCTL2_IDO_CMP_EN 0x0200

The fact is that I’m not sure whether it’s the better (and cleaner) way to solve the problem. I contacted RIFFA’s developers and they show me the differences between the both (“NVidia” and “Linux” kernel) pci_regs.h files :

diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index baa7852..c3cc01d 100644
--- a/include/uapi/linux/pci_regs.h (3.11)
+++ b/include/uapi/linux/pci_regs.h (3.12)
 #define PCI_EXP_DEVCTL2                40      /* Device Control 2 */
-#define  PCI_EXP_DEVCTL2_ARI           0x20    /* Alternative Routing-ID */
-#define  PCI_EXP_DEVCTL2_IDO_REQ_EN    0x0100  /* Allow IDO for requests */
-#define  PCI_EXP_DEVCTL2_IDO_CMP_EN    0x0200  /* Allow IDO for completions */
-#define  PCI_EXP_DEVCTL2_LTR_EN                0x0400  /* Enable LTR mechanism */
-#define  PCI_EXP_DEVCTL2_OBFF_MSGA_EN  0x2000  /* Enable OBFF Message type A */
-#define  PCI_EXP_DEVCTL2_OBFF_MSGB_EN  0x4000  /* Enable OBFF Message type B */
-#define  PCI_EXP_DEVCTL2_OBFF_WAKE_EN  0x6000  /* OBFF using WAKE# signaling */
-#define PCI_EXP_DEVSTA2                42      /* Device Status 2 */
+#define  PCI_EXP_DEVCTL2_ARI   0x20    /* Alternative Routing-ID */
+#define  PCI_EXP_IDO_REQ_EN    0x100   /* ID-based ordering request enable */
+#define  PCI_EXP_IDO_CMP_EN    0x200   /* ID-based ordering completion enable */
+#define  PCI_EXP_LTR_EN                0x400   /* Latency tolerance reporting */
+#define  PCI_EXP_OBFF_MSGA_EN  0x2000  /* OBFF enable with Message type A */
+#define  PCI_EXP_OBFF_MSGB_EN  0x4000  /* OBFF enable with Message type B */
+#define  PCI_EXP_OBFF_WAKE_EN  0x6000  /* OBFF using WAKE# signaling */

They also proposed me to, either keep my fix or use the actual kernel version (with DEVCTL2).

However, they don’t seem completely sure about the “DEVCTL2” keyword, is it just a rewriting of the defines or is it a particular functionality ?

Was this a native compile on JTX1, or cross-compile? Native compile has issues due to 32-bit user space and 64-bit kernel space, so cross-compile is more or less required. For cross-compile you’ll want the latest 5.2 version of Linaro, rather than 4.8…5.3 should also work, but I know 5.2 works well (you’ll need the full aarch64-linux-gnu and gcc from arm-linux-gnueabihf). See:
https://releases.linaro.org/components/toolchain/binaries/

Hi,

It was a native compile. I already tried cross-compile with Linaro (4.8 ? don’t remember) from Host and kernel 3.10.67 sources for the Xillybus driver but I had problem with the kernel name. It missed “-g458d45c” at the end, so I laid that aside for now.

My “problem” is more about the pci_regs.h file and its #define names. I understood that the L4T kernel is a version between the 3.11 and 3.12 but these ones have the “DEVCTL2” word in their #define names.

The v4.8 compiler would be a problem, support for kernel space ARMv8a is more or less a “v5+ thing”. The 32-bit side would be an ARMv8 32-bit within the kernel.

The “-g458d45c” is not specific to Jetson, this is the CONFIG_LOCALVERSION needing to be set…it is how all Linux systems find modules. If the kernel source code is “3.10.67”, and no CONFIG_LOCALVERSION, then “uname -r” becomes “3.10.67”. If 3.10.67 has CONFIG_LOCALVERSION set to “-g458d45c”, then “uname -r” becomes “3.10.67-g458d45c”. A kernel searches for modules depending on “uname -r”. The location is:

/lib/modules/$(uname -r)

Under “make menuconfig” see:

General setup --->
   Local version - append to kernel release

Mismatch between expected “uname -r” of kernel or modules will be a problem.