I am working on a linker module related to GPU compilation and need to deepen my understanding of how linking works specifically for device code (NVIDIA/AMD GPUs).
My questions are:
1)Linker scripts for GPU device code:
Does a linker script exist for device code during GPU compilation? If yes, is it possible to view or modify it? How are addresses assigned to device symbols during the linking process?
2)Host and device code compilation with CUDA:
According to NVIDIA’s CUDA compiler documentation, host and device code compilation happens together. How exactly does this process work internally? How does nvlink handle the linking of device code and symbol resolution?
3)Address mapping and sections for GPU device memory:
I inspected the /proc//maps for a process using NVIDIA devices and found various address mappings related to NVIDIA device files (e.g., /dev/nvidia0, /dev/nvidiactl, /dev/nvidia-uvm). Here is an example snippet:
$ cat /proc/1743261/maps | grep nvidia
201200000-201400000 rw-s 00000000 00:05 614 /dev/nvidiactl
204000000-204200000 rw-s 00000000 00:05 614 /dev/nvidiactl
204200000-204400000 rw-s 204200000 00:05 627 /dev/nvidia-uvm
324200000-324400000 rw-s 00000000 00:05 616 /dev/nvidia0
324400000-324600000 rw-s 00000000 00:05 616 /dev/nvidia0
324600000-324800000 rw-s 00000000 00:05 616 /dev/nvidia0
324800000-324a00000 rw-s 00000000 00:05 616 /dev/nvidia0
324a00000-324c00000 rw-s 00000000 00:05 616 /dev/nvidia0
324c00000-324e00000 rw-s 00000000 00:05 616 /dev/nvidia0
324e00000-325000000 rw-s 00000000 00:05 616 /dev/nvidia0
325000000-325200000 rw-s 00000000 00:05 616 /dev/nvidia0
325200000-325400000 rw-s 00000000 00:05 616 /dev/nvidia0
325400000-325600000 rw-s 00000000 00:05 616 /dev/nvidia0
325600000-325800000 rw-s 00000000 00:05 616 /dev/nvidia0
325800000-325a00000 rw-s 00000000 00:05 616 /dev/nvidia0
325a00000-325c00000 rw-s 00000000 00:05 616 /dev/nvidia0
325c00000-325e00000 rw-s 00000000 00:05 616 /dev/nvidia0
325e00000-326000000 rw-s 00000000 00:05 616 /dev/nvidia0
326000000-326200000 rw-s 00000000 00:05 616 /dev/nvidia0
327400000-327600000 rw-s 00000000 00:05 614 /dev/nvidiactl
327a00000-327feb000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8c800000-fffd8e6b7000 r-xp 00000000 103:04 52453027 /usr/lib64/libnvidia-ptxjitcompiler.so.570.124.06
fffd8e6b7000-fffd8e6c6000 —p 01eb7000 103:04 52453027 /usr/lib64/libnvidia-ptxjitcompiler.so.570.124.06
fffd8e6c6000-fffd8ec54000 r–p 01eb6000 103:04 52453027 /usr/lib64/libnvidia-ptxjitcompiler.so.570.124.06
fffd8ec54000-fffd8ec59000 rw-p 02444000 103:04 52453027 /usr/lib64/libnvidia-ptxjitcompiler.so.570.124.06
fffd8eddf000-fffd8ede0000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede0000-fffd8ede1000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede1000-fffd8ede2000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede2000-fffd8ede3000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede3000-fffd8ede4000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede4000-fffd8ede5000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede5000-fffd8ede6000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede6000-fffd8ede7000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede7000-fffd8ede8000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede8000-fffd8ede9000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8ede9000-fffd8edea000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8edea000-fffd8edeb000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8edeb000-fffd8edec000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8edec000-fffd8eded000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8eded000-fffd8edee000 rw-s 00000000 00:05 614 /dev/nvidiactl
fffd8edee000-fffd8edef000 rw-s 00000000 00:05 614 /dev/nvidiactl
ffff9f7ae000-ffff9f7be000 -w-s 00000000 00:05 616 /dev/nvidia0
ffff9fd12000-ffff9fd13000 r–s 00000000 00:05 614 /dev/nvidiactl
ffff9fd13000-ffff9fd23000 -w-s 00000000 00:05 616 /dev/nvidia0
4)How can I map these address ranges back to specific sections or symbols? In other words, how do I determine which address is assigned to which section of device or host code during or after linking?
5)Any pointers, documentation references, or explanations about how linking and symbol resolution works in GPU device compilation (CUDA or AMD) would be very helpful.
Thank you!