There seems to be something missing in the Driver API when it comes to creating a Module from a fatbin file.
cuModuleLoadFatBinary doesn’t have any options. Less of a problem is that there is no function to load a file in fatbin format, so the programmer has to manually read the file into memory (or use something like boost/interprocessiostreams to memory map the file portably); c++ has no standard library for memory mapped files, unfortunately.
The Toolkit doc says that cuModuleLoadData[Ex] and cuModuleLoadFile accept fatbin format. This begs the question as to whether cuModuleLoadFatBinary is obsolete. If the fatbin format is acceptable, then there’s the same difficulty that in order to provide options, one has to read / map the file into memory and give that image to the API.
I also wonder if any of the options are applicable to fatbin format. Obviously the compile options are N/A. But things like CU_JIT_WALL_TIME, or CU_JIT_TARGET, would be useful. Does the API use CU_JIT_TARGET to select which binary to extract from the fatbin?
A further, minor, question is why there’s no Link function for fatbins. Is this because fatbins are always fully linked and not relocatable? Will cuLinkAddFile/Data accept fatbin format? (see next comment)
I noticed the CUjitInputType parameter for cuLinkAdd(Data|File). This includes CU_JIT_INPUT_FATBINARY, which implies that a fatbin can be composed of various types of items.
I think the behavior of options with various API calls can be described as (please correct any errors):
There are three places where options are supplied:
cuModuleLoadDataEx, or Load.
cuModuleLinkCreate, or Link.
cuModuleLinkAdd*, or LinkAdd.
There are two flavors of options:
- Compiler options. These are applicable only when a PTX file (or something containing one) is being compiled. Example, CU_JIT_MAX_REGISTERS. The API document lists these as “compiler only”.
They may be supplied to the Linker via Link. In this case, they apply to any LinkAdd calls as a default.
Each Add call supplies compiler options that apply to that file only, and override any options of the same name given to the Link.
Compiler options are also valid for Link.
- Linker options. Given only to Link. Example, CU_JIT_WALL_TIME. They apply regardless of the type of inputs being later processed. Some of these are used by LinkAdd also, such as info and error logs and wall time. They are documented as “compiler or linker”, which, in my opinion, is misleading and should be corrected.
They are not intended to be passed directly to LinkAdd, or Load. Doing so is not an error but produces unspecified results.
I am curious as to whether (and how) CU_JIT_INPUT_OBJECT and CU_JIT_INPUT_LIBRARY actually work with cuLinkAdd*. Is it looking for an actual host object (.obj or .o) or archive (.lib or .a) file? Is it looking for a particular named resource embedded in the file? In the case of an archive, which archive member does it look in?